Path 1 · Small startup
Small startup — run a ~100–250B model on a budget
NVIDIA H200 SXM — the hardware in this build. Image: NVIDIA
- A ~100–250B model is the sweet spot for a startup: smart enough to be useful, small enough to own the hardware.
- GPU memory is the workbench — the whole model must fit here or it doesn't run at all. Regular RAM cannot substitute: it sits on a link far too slow for the GPU, which would spend nearly all its time waiting instead of computing.
- A 235B model needs 235 GB + 20% = 282 GB. No single card holds that — but two H200s (141 GB each) pooling memory in one server do.
- Budget matters, and quantization is your friend: Quantization means storing the model's numbers less precisely so it fits in less memory — 8-bit halves it, 4-bit halves it again, with some quality loss. At 4-bit, the same model fits on one H200 — half the price, some quality loss.
Recommended build: 2× NVIDIA H200 SXM
- Total GPU memory: 282 GB (combined)
- Total price: $70,000
- Total power: 1,400 W — about 1.17 home; one day (33.6 kWh) drains about 0.37 EV battery
Fits Qwen3-235B (needs exactly 282 GB) and anything smaller.
Honest note: for Qwen3-235B this fits with ZERO room to spare — 282 GB needed, 282 GB provided. Quantize to 8-bit for breathing room, or step up to 3× H200 (423 GB). Quantization means storing the model's numbers less precisely so it fits in less memory — 8-bit halves it, 4-bit halves it again, with some quality loss.
Request a quote for this build
