August 12, 2026
Pretraining a foundation model, or running continued pretraining on domain-specific data at meaningful scale, is not a job you provision on-demand and monitor casually. It is a multi-week or multi-month compute campaign where the infrastructure needs to behave predictably on day 90 the same way it did on day one. That requirement narrows the infrastructure options considerably.
The conversation about training infrastructure often starts with peak throughput, tokens per second, GPU utilization, MFU (model FLOPs utilization). Those numbers matter, but they are not what determines whether a sustained training campaign succeeds operationally.
What matters over a 12-month compute commitment is a different set of properties: consistency of throughput over weeks, not benchmarks; stability of the software environment across the full job duration; predictable storage I/O for continuous checkpoint writes; and a fixed cost structure that does not shift with spot market availability or egress volume.
Shared cloud infrastructure is optimized for flexibility, the ability to provision and deprovision resources quickly, scale up and down with demand, and pay only for what you use. Those properties are genuinely useful for workloads that need them. They are actively counterproductive for sustained pretraining, where the operational requirement is the opposite: predictable, dedicated capacity over a fixed term, with no competing workload on the same fabric and no metered cost that accumulates with usage.
The B300's memory architecture changes the practical model size ceiling for single-node and small-cluster configurations in ways that matter directly to model labs.
At 288 GB of HBM3e per GPU, a single 8-GPU B300 node holds approximately 2.3 TB of total GPU memory. A 70B parameter model in BF16 fits entirely on a single GPU. A 200B parameter model fits within a single node. For continued pretraining runs on models in the 70B to 200B range, the size class where most domain-specialized foundation model work is currently concentrated, a single B300 node eliminates the inter-node tensor parallelism overhead that currently limits throughput on H100 configurations, where those same models require two to four nodes.
The compute picture follows the same logic. At 15 petaFLOPS of dense FP4 per GPU, the B300 delivers roughly 18 times the FP8 throughput of the H100. For labs running long training campaigns where the total token budget is fixed and time-to-completion directly affects the research roadmap, that throughput advantage translates to faster iteration cycles, more training runs completed per quarter, shorter feedback loops between data experiments and model evaluation.
The fifth-generation Tensor Cores also support automatic mixed precision across FP4, FP8, and BF16 within a single training job, managed by the NVIDIA Transformer Engine. For labs training on heterogeneous data mixtures, where different data types benefit from different precision levels, this reduces the manual precision management overhead that has historically required specialized tooling.
For model labs planning a 12-month training commitment, cluster configuration is not a provisioning decision, it is an architectural one. The configuration you start with determines the communication topology, the checkpoint strategy, and the parallelism approach for the full duration of the campaign.
A single B300 NVL8 node, 8 GPUs, 2.3 TB GPU memory, 14.4 TB/s NVSwitch fabric, handles models up to approximately 200B parameters in BF16 without requiring inter-node tensor parallelism. Data parallelism across multiple NVL8 nodes scales cleanly for training runs where the model fits within a single node but the data volume requires parallel processing across multiple GPUs.
For models above 200B parameters, or for labs running pipeline-parallel configurations across multiple nodes, the B300 NVL16, two baseboards connected through an extended NVSwitch fabric, 16 GPUs, 4.6 TB total GPU memory, keeps the communication within the NVSwitch domain rather than pushing it to the inter-node network. That distinction matters for training efficiency: NVSwitch communication at 28.8 TB/s aggregate fabric bandwidth operates at a different latency profile than InfiniBand, and keeping tensor-parallel communication within the NVSwitch domain reduces the communication overhead that limits MFU on multi-node tensor-parallel configurations.
Between nodes, ConnectX-8 SuperNICs at 800 Gb/s per GPU handle the inter-node fabric for pipeline and data parallelism. For a lab running continued pretraining with data parallelism across four to eight B300 nodes, the inter-node bandwidth is not typically the binding constraint, but for large-scale pipeline-parallel configurations, it is.
The technical properties of the B300 are fully realized only on dedicated infrastructure. On shared cloud, the variables that degrade sustained training, NVLink contention from neighboring tenants, inconsistent storage I/O during checkpoint writes, driver updates applied mid-job, egress costs accumulating on every checkpoint write, interact with the hardware in ways that no software-level optimization fully compensates for.
On dedicated bare metal on a 12-month term, those variables do not exist. The cluster is yours. The NVSwitch fabric is yours. The storage throughput is consistent and uncontested. The software environment, driver versions, CUDA toolchain, NCCL configuration, is fixed for the duration of the term and updated only on your schedule.
The 12-month commitment is not a constraint, it is the operational model that sustained training campaigns require. A lab running a multi-month pretraining campaign needs to know its compute capacity twelve weeks in advance, not provision reactively based on spot availability. The training schedule, the checkpoint strategy, the data pipeline throughput, and the team's research roadmap are all built on the assumption of consistent, predictable infrastructure. Dedicated bare metal on a fixed term provides that foundation. Shared cloud, by design, does not.
Checkpoint economics also change materially on dedicated infrastructure. A 70B parameter model checkpoint in BF16 is approximately 140 GB. A sustained training campaign writing checkpoints every 500 steps over six months generates checkpoint volume that, on metered cloud infrastructure, accumulates into a significant unplanned cost. On dedicated bare metal with unmetered bandwidth, checkpoint writes have no marginal cost regardless of frequency or volume.
1Legion operates dedicated B300 bare metal clusters for AI model labs running sustained training campaigns, full-server allocation, no shared tenancy, no egress fees, unmetered bandwidth, on 12-month dedicated terms.
If you are planning a pretraining campaign or evaluating dedicated infrastructure for a long-duration training commitment, talk to an engineer about your cluster requirements today. Talk to an Engineer here.
