Your GPU memory (VRAM), GB On a Mac or a CPU box with offload, use your usable unified/system memory instead.

Context length (k tokens) How much you load at once. Longer context eats more memory.

Everything is computed in your browser — no inputs are sent anywhere. Numbers are estimates for planning, not a guarantee; actual use depends on your runtime (llama.cpp, vLLM, Ollama), batch size, and driver overhead.

How the estimate works

Loading a model into memory is mostly three things added together, and only the first is exact:

Weights — the model file itself. This is just the parameter count times the bytes each weight takes at your quantization, so it is the precise part. A rule of thumb for the common Q4_K_M quant is parameters × ~0.6 GB (an 8B model is about 4.8 GB, a 70B about 42 GB).
KV cache — the running memory the model uses to "remember" the conversation, which grows with context length. This is an estimate: it depends on the model's layer count and attention design (this tool assumes a modern grouped-query layout with a 16-bit cache), and a runtime can shrink it by quantizing the cache.
Overhead — CUDA/Metal context, activations, and framework buffers. A small allowance, also approximate.

In short, the calculator works out total ≈ (params × bytes-per-weight) + KV cache + overhead, where the KV cache ≈ layers × context × 4 KB for a 16-bit grouped-query cache (the 4 KB per layer per token already bundles both the keys and the values), and overhead is a small fixed allowance (a fraction of the weights, with a floor). Only the first term is exact; the other two are the estimates, and a different runtime or a quantized cache can move them.

The weights term dominates for short context; the KV cache is what makes a model that "fit" at 4k suddenly overflow at 32k. That is why the calculator asks for context separately, and why a long-context plan needs headroom.

Quantization quick reference

These are the bytes-per-weight the calculator uses for the weights term, per billion parameters, for the common GGUF quants — a slightly conservative rounding of the real quant sizes, so it errs toward needing a bit more memory rather than less. Lower bits trade a little quality for a lot of memory.

Quant	Bits/weight (approx)	GB per 1B params	7B weights	70B weights
Q4_K_M	~4.5–4.8	~0.60	~4.2 GB	~42 GB
Q5_K_M	~5.5	~0.65	~4.6 GB	~46 GB
Q6_K	~6.6	~0.82	~5.7 GB	~57 GB
Q8_0	~8.5	~1.0	~7.0 GB	~70 GB
FP16 / BF16	16	~2.0	~14 GB	~140 GB

These track the standard GGUF quant conventions (a real Q4_K_M GGUF of an 8B model lands near 4.9 GB; Q8_0 near 8 GB) and the widely-used “×0.6 for Q4_K_M” rule of thumb. Actual files vary a little by model and quant tool — see the sources below.

Popular local models (parameter counts)

This is the list the preset dropdown reads from — a snapshot of models people commonly run locally, with the parameter count that drives the weights math. It is dated on purpose; new models ship constantly, so the calculator is built to work from raw parameters for anything not listed here.

Model	Parameters	Notes
Llama 3.2 3B	3B	Small, fits tight hardware
Llama 3.1 8B	8B	Common 8 GB-class pick
Llama 3.3 70B	70B	~42 GB of weights at Q4; ~48 GB total at 8k
Qwen2.5 7B	7B	Dense
Qwen2.5 14B	14B	Dense
Qwen2.5 32B	32B	Dense
Qwen2.5 72B	72B	Dense
Mistral 7B	7B	Fast, light
Mistral Small 24B	24B	Dense
Gemma 2 9B	9B	Dense
Gemma 2 27B	27B	Dense
Phi-4 14B	14B	Strong reasoning for its size
DeepSeek-R1-Distill 32B	32B	Dense distill
Qwen3 235B (MoE)	235B total	MoE: all weights load (~22B active)
DeepSeek-R1 671B (MoE)	671B total	MoE: multi-GPU / huge memory
Kimi K2.7-Code (MoE)	1T total	MoE: all weights load (~32B active); ~595 GB repo at native INT4
GLM-5.2 (MoE)	744B total	MoE: all weights load (~40B active)

Model list updated: June 16, 2026. Parameter counts from the models' public model cards and the Ollama library. For anything newer, use the "will it fit" tab and type the parameter count.

Sources

Hugging Face — GGUF format documentation (quantization format and naming).
llama.cpp (the GGUF quant types the bytes-per-weight figures follow).
Example GGUF model card (Llama 3.1 8B) showing real per-quant file sizes.
Ollama model library (model names, parameter counts, and download sizes).

VRAM, system RAM, and unified memory

The calculator is written around GPU VRAM because that is the hard ceiling for fast inference, but the same numbers map onto other setups. On a CPU-only or Xeon box, the model loads into system RAM and runs slower; tools like Ollama and llama.cpp can also split a model between GPU and RAM (offload), so "12 GB" can mean GPU VRAM plus spillover. On Apple Silicon, there is no separate VRAM — the GPU shares unified memory with the system, so use your total usable memory as the figure. If you are sizing an older or cheaper machine for this, the trade-offs are in our guides on running a local LLM on cheap old hardware and a beginner Ollama setup.

What this can't tell you

A model fitting in memory is necessary, not sufficient. It does not promise the speed you want — tokens per second depend on memory bandwidth and the chip, not just capacity — and it says nothing about quality, which a heavy quant can quietly erode. Treat a "fits" result as permission to try, then confirm with your actual runtime. Mixture-of-experts models are the common trap: only a fraction of the parameters are active per token, but the whole model still has to be in memory, so size them by total parameters, not the active count.

Method & sources. Weights are computed from your parameter count times the bytes-per-weight of the chosen quantization, using published GGUF sizes and the standard Q4_K_M rule of thumb. KV cache and overhead are deliberately labeled estimates that vary by model architecture and runtime. GearBriefly built this tool, takes no affiliate revenue from it, and shows the formula so you can check the numbers. Quantization sizes: published GGUF model sizes and llama.cpp quant conventions. Model parameters: the models' public model cards and the Ollama model library.