July 16, 2026
The B300 entered mass production in June 2026. GB300 shipments are projected to grow roughly 129 percent year over year, with Microsoft, Amazon, and Meta among the lead adopters. The question for most AI teams is not whether Blackwell Ultra is the dominant infrastructure generation, it is how to access it, and on what terms.
Those terms matter more than most infrastructure conversations acknowledge.
For most of the period between 2022 and 2025, the H100 was the answer to most serious AI infrastructure questions. Training, inference, fine-tuning, the H100 handled all of it, and the ecosystem around it, CUDA toolchains, serving frameworks, operator knowledge, matured to match.
Two things changed that equation. First, model scale crossed thresholds the H100 was not designed for. Reasoning models with extended chain-of-thought generation, 200B+ parameter architectures, long-context serving at scale, these workloads push against the H100's 80 GB memory ceiling in ways that tensor parallelism and quantization can partially address but never fully resolve. Second, inference economics became the primary driver of infrastructure decisions as AI moved from research into production. Cost-per-token, fleet size, and throughput per watt are now the metrics that determine whether a deployment is viable, not just whether it runs.
The B300 was built for this moment. Understanding what it actually delivers, rather than what the headline specs suggest, is where the infrastructure planning decision gets concrete.
The B300 ships with 288 GB of HBM3e memory per GPU and 8 TB/s of memory bandwidth. For teams familiar with H100 infrastructure, the practical consequence is that model parallelism requirements change entirely. A 70B parameter model in FP16, which requires two H100s at minimum, fits on a single B300 with roughly 148 GB remaining for KV cache and batch headroom. A 200B+ model fits across a single 8-GPU B300 node without the tensor parallelism overhead that currently limits throughput at that scale.
On compute, the B300 delivers 15 petaFLOPS of dense FP4 per GPU. Independent benchmarks from SemiAnalysis place Blackwell Ultra systems at up to 50 times higher throughput per megawatt and up to 35 times lower cost per token than Hopper-generation hardware for low-latency agentic workloads. On DeepSeek-R1 in MLPerf Inference v6.0 results from April 2026, Blackwell Ultra systems delivered 2.5 million tokens per second, 2.7 times higher than Blackwell Ultra's own debut submissions six months prior, driven by TensorRT-LLM software updates.
That last figure is worth holding: the hardware's realized performance is still improving through software. Teams deploying B300 infrastructure now are not buying a static capability, they are buying into a platform that continues to improve as NVIDIA's inference software stack matures.
The migration path from H100 is also cleaner than prior generation transitions. The B300 runs on the same CUDA toolchain. Existing workloads run without code changes. FP4 support requires TensorRT-LLM 0.15 or higher or vLLM with FP4 quantization enabled, an inference stack update, not a pipeline rewrite. For more detail on the specific workload-by-workload migration decision, see our post on migrating from H100 to B300.
The way you access B300 infrastructure determines how much of its technical capability your workloads actually use. This is not a marginal distinction.
On shared cloud infrastructure, multi-tenant GPU environments operated by hyperscalers or GPU cloud providers, the B300's peak capability is degraded by variables the hardware itself cannot compensate for. NVLink bandwidth is contested when neighboring tenants are active. Storage I/O is inconsistent during checkpoint writes or dataset loading. Egress fees apply to every model artifact, training output, and inference result that leaves the node. And the specific driver and software configuration your workload was validated on can shift without notice.
These are structural properties of shared infrastructure, not edge cases. For a GPU as capable as the B300, 8 TB/s of memory bandwidth, 15 petaFLOPS of FP4 compute, any external contention becomes a proportionally larger drag on realized throughput. A workload achieving 70 percent GPU utilization on shared infrastructure can typically reach 90 percent or above on dedicated nodes. Across a long training run or a high-traffic inference deployment, that gap is real compute budget.
On dedicated bare metal, none of those variables apply. The GPU's full resource profile belongs to your workload. No shared tenancy. No egress fees. No noisy neighbors on the interconnect. The configuration you validate before deployment is the configuration that runs in production.
For teams planning B300 infrastructure for LLM training specifically, the bare metal argument connects directly to the architectural gains described above, the B300's memory capacity advantage is only fully realized when the interconnect bandwidth is not contested. For inference at scale, consistent latency under concurrent load requires a predictable infrastructure layer. Dedicated bare metal provides both.
For a deeper look at how the B300's architecture applies specifically to production inference workloads, see our post on B300 for production inference.
