Skip to content
Trusted US Based Fiber Optics Partner
400G DR4

NVIDIA H100 SuperPOD Optics: A 256-GPU Fabric Case Study

256-GPU NVIDIA H100 SuperPOD topology — 32 DGX H100 nodes, 8 leaf + 4 spine Quantum-2 QM9700, 256× 400G NDR InfiniBand links, rail-optimized non-blocking fat-tree. Vitex AI data center case study.

Most transceiver orders are routine. This one wasn't — not because of its size, but because of what it turned out to be. It arrived on a Saturday, from a company Vitex had never sold to before, with two destinations on opposite U.S. coasts and a build deadline already counting down. And folded into the part numbers was a complete 256-GPU H100 compute fabric.

Customer
First-time cloud GPU infrastructure provider
The Build
256-GPU H100 fabric · Quantum-2 + ConnectX-7
Footprint
Two U.S. sites · opposite coasts
Window
17 days · inquiry to both shipments

The Order

Three line items, one 256-GPU fabric

There were two orders. The first was easy to place — 400G QSFP-DD DR4 transceivers and matched MPO fiber for a connectivity expansion at one site, same-generation optics going into a deployment that already existed. The second was the one worth slowing down on:

Order 2 · as it arrivedSaturday
01TWIN-PORT 800G OSFP TRANSCEIVERQTY 128
02SINGLE-PORT 400G OSFP TRANSCEIVERQTY 256
03MPO SMF ASSEMBLY · CUT TO 5 LENGTHSQTY 256
2 ship-to addresses · opposite U.S. coasts · deadline running

Those quantities aren't arbitrary. Run the arithmetic and they resolve into something specific and recognizable.

The Architecture

Inside the SuperPOD: 8 leaf, 4 spine, one 400G path per GPU

A SuperPOD scalable unit is NVIDIA's reference recipe for 256 GPUs: thirty-two DGX H100 nodes, eight ConnectX-7 SuperNICs per node — one per GPU — wired into a two-tier Quantum-2 InfiniBand fabric. The optics are the part of the recipe that arrives on a pallet. This BOM was that part, exactly:

Line item Qty Seats in Resolves to
Single-port 400G OSFP 256 ConnectX-7 SuperNICs One NIC per GPU → 256 GPUs · 32 nodes
Twin-port 800G OSFP 128 Leaf access cages · 16 per leaf 2× 400G NDR each → 256 leaf-side ports
MPO single-mode assemblies 256 · 5 lengths Leaf cage ↔ NIC One fiber path per GPU

Table 1 — How each line item in the order maps to its seat in the fabric, and what the quantities resolve to.

Rail-optimized sets the wiring plan for every row of that table: NIC 1 of every node lands on leaf switch 1, NIC 2 on leaf 2, and so on — eight rails, eight leaf switches — so the bulk of collective traffic stays one hop away on a single leaf instead of crossing the spine:

EIGHT RAILS — ONE PER NIC DGX H100 8× CONNECTX-7 NIC 1NIC 2 NIC 3NIC 4 NIC 5NIC 6 NIC 7NIC 8 LEAF 1 · QM9700 LEAF 2 · QM9700 LEAF 3 · QM9700 LEAF 4 · QM9700 LEAF 5 · QM9700 LEAF 6 · QM9700 LEAF 7 · QM9700 LEAF 8 · QM9700 × 32 NODES same pattern, every node
One node's eight rails: NIC k terminates on Leaf k — repeated identically across all 32 nodes.

Every one of the thirty-two nodes repeats that same fan-out. Above the leaves, four spines keep the arithmetic honest:

GPU side
32 nodes × 8 NICs = 256× 400G down
Spine side
8 leaves × 32 uplinks = 256× 400G up
Oversubscription
Down = up → 1:1, non-blocking

Lay it out and the order resolves into the complete GPU-to-leaf access layer of the fat-tree. The leaf-to-spine tier and its optics sat outside this order's scope; knowing which tier of the fabric a BOM belongs to is part of reading the BOM.

2-tier rail-optimized fat-tree — 256-GPU NVIDIA H100 SuperPOD: 8 leaf and 4 spine Quantum-2 switches, one 400G path per GPU
Figure 1 — The same three line items, drawn out: the GPU-to-leaf access layer of a 2-tier rail-optimized fat-tree — 8 leaf + 4 spine Quantum-2 — one 400G path per H100 GPU.

Reading a clean piece of network engineering off a parts list is the first thing a good optics supplier owes the customer — and on a fabric this specific, it's also where the time is either saved or lost.

The Diligence

Twin-port IHS in the leaf cages, single-port RHS on the NICs

A bill of materials like this is exactly where an order can quietly go wrong — not on the part numbers, on the cage at each end of the link:

End of the link Cage accepts In this order
Quantum-2 leaf access cage Finned IHS only 128× twin-port 800G IHS
ConnectX-7 NIC cage Flat-top RHS only 256× single-port 400G RHS

Table 2 — Each end of the link accepts only one heat-sink form factor: finned IHS in the Quantum-2 leaf cages, flat-top RHS in the ConnectX-7 NIC cages.

Same OSFP family, physically non-interchangeable: a finned module won't seat in a NIC cage, and a flat-top module won't cool correctly in a switch cage. On a build deadline, a single mismatched form factor becomes modules sitting on a dock while the order is re-spec'd and re-shipped — measured in weeks, not dollars.

So before the final quote went out on the larger order, Vitex confirmed the exact platform on both ends — switch model and NIC model — and matched every module to its cage.

Worth knowing: mixed-form-factor fabrics — IHS in the leaf cages, RHS on the NICs, within one build — are where AI data center optics orders most often slip. Confirming the platform before quoting is what keeps a wrong-cage shipment from showing up on the dock.

That verification step is also part of why the timeline reads the way it does. An order of this size and specificity — two sites, mixed form factors, a brand-new account — would commonly quote at several weeks, and a good share of that lag usually comes from the platform back-and-forth happening after the purchase order rather than before it. Here the product research and the platform check ran up front, in parallel with onboarding and pricing, so the quote that went out was already the correct one. There was no second pass.

The Execution

U.S. stock, platform lock, parallel onboarding

Three things, mostly — and the point is less what they were than that they ran at the same time instead of one after another. The smaller site's 400G optics were already in stock in New Jersey, so that order shipped the same business day its PO cleared. The platform was verified before the quote, so there was no re-quote cycle. And because the customer was new, vendor onboarding — the W-9, the terms, the paperwork that usually waits until after a quote is accepted — was handled alongside the quoting rather than after it.

Three levers behind the 17-day window: U.S. inventory ready to ship same day, platform verified before final quote, onboarding run in parallel
Figure 2 — None of the three is dramatic alone. Run in parallel rather than in series, they're the difference between seventeen days and several weeks.
The procurement clock and the engineering clock have to run together. In series, they miss the deadline almost every time.

The Result

Two POs in five days, both shipments in seventeen

Five days after the Saturday inquiry, both purchase orders were in. The smaller site's order had already gone — shipped same-day from stock. The H100 fabric followed as a consolidated, multi-carton release on priority overnight, and seventeen days after the first email both shipments were in transit to their respective coasts.

Engagement timeline: Day 1 Saturday inquiry, Day 5 two POs signed, Day 12 400G site ships from U.S. stock, Day 17 H100 fabric ships consolidated priority overnight
Figure 3 — Five days from a Saturday inbound to two signed POs; twelve more to the consolidated H100 fabric release.

The Takeaway

Confirm the platform, quote against real stock

Two things make an engagement like this work, and neither is price.

01
Stock that's real

The right optics physically in the country, on the shelf, in the form factors a current NVIDIA build actually uses — so that "in stock" means shipped today, not sourced in a month.

02
The build, read first

Reading the build correctly before quoting it — so that the box which arrives is the box that fits.

Vitex has supplied optical transceivers, DAC/AOC/AEC/ACC cables, and fiber for over 23 years from Hackensack, New Jersey, and the current focus is mostly 400G, 800G, and early 1.6T optics for AI data centers. The same approach applies whether it's a single top-up reel or a 256-GPU fabric spread across two coasts: confirm the platform, quote against real stock, and move on the build clock the customer is already running.

Sourcing 400G or 800G optics under a build deadline?

Send the BOM and the NVIDIA platform you're terminating against — switch and NIC both. The Vitex engineering team will confirm the form factors, quote against current U.S. stock, and give you a realistic ship date before you commit to a PO.

Talk to a Vitex Engineer →

Leave A Comment

Please note, comments need to be approved before they are published.

Talk to an Optical Engineer

Get engineering answers before you commit

Share your BOM, validate compatibility, or sanity-check 400G/800G designs. Get fast, practical guidance from US-based fiber optics engineers.