GPU resource calculator

How the estimate is made

• VRAM is parameters × bytes per parameter (FP32 = 4, FP16/BF16 = 2, INT8 = 1) × an overhead factor for KV cache, activations, and framework memory: 1.5× for LLMs, 1.2× for TTS, 1.3× for vision, 1.6× for multimodal.
• Per-GPU throughput starts from a reference of 60 requests/second for a 7B FP16 model on a V100 and scales inversely with model size, then by output length relative to 256 tokens (clamped to 0.25–4×). TTS and vision models use fixed conservative baselines.
• GPU count is peak RPS divided by per-GPU throughput, rounded up.
• Cost picks one instance per provider from the VRAM tier your model lands in, multiplies the hourly Linux list price by 730 hours and your traffic pattern's utilization, and shows on-demand, 1-year reserved, and a typical spot discount for that provider.

Frequently asked questions

How do the prices stay current?

Hourly rates are Linux on-demand list prices: AWS in us-east-1, GCP in us-central1, and Azure in East US. They were last checked in October 2026. The page does not call the cloud price APIs, so a number stays until that check is repeated and the date is updated.

On-demand in the table is that hourly rate times 730 hours times your traffic pattern, so Business Hours is 40% of a full month. Reserved applies a typical one-year discount. Spot applies a typical discount for that provider, not today's spot market. Storage and data transfer are not included. Use the figure to pick an instance family, then price the real deployment in the provider's calculator.

How accurate is this GPU calculator?

It is a first-order sizing model. Real throughput depends on your serving stack (vLLM, TensorRT-LLM, Triton), batching, quantization, prompt length, and tensor parallelism. Use it to pick an instance family and a budget range, then benchmark with your actual model before you commit.

Which cloud providers and GPUs are covered?

AWS, Google Cloud, and Microsoft Azure, with T4, L4, A10G, V100, A100, and H100 instance families. The data file is open; prices and baselines are easy to update.

What model types can I size?

Large language models, text-to-speech, TTS with an LLM front-end and SNAC decoder, vision models, and multimodal models.

How do I cut GPU cost?

Quantize to INT8 or FP8 where quality allows, batch requests, use spot capacity for anything that can be retried, commit to reserved pricing for the baseline load, and auto-scale the rest. The traffic pattern selector shows how much of the bill is utilization rather than hardware.