VRAM calculator
Enter your GPU and a model. It works out weights, KV cache and overhead, and tells you whether it fits — or what to change.
How this is calculated
Three things occupy VRAM when you run a local model.
Weights are parameter count multiplied by bytes per parameter. FP16 is two bytes; quantised formats pack it down, and the effective rate for GGUF k-quants is a little above the nominal bit width because some tensors stay at higher precision.
KV cache holds the attention keys and values for every token
in context. It is 2 × layers × kv_heads × head_dim ×
context × 2 bytes. Models with grouped-query attention have far fewer
KV heads than attention heads, which is why a modern 8B model uses a fraction of
the cache an older one would.
Overhead is the CUDA context, activations and buffers. It varies by runtime; this assumes roughly 0.8 GB, which is typical for llama.cpp.
Treat the output as a close estimate, not a guarantee. Architecture figures for the presets are approximate, real allocators fragment, and runtimes differ. If a result lands within about 10% of your card's capacity, assume it will not fit.
Why we built this
Varven runs a language model on your own machine to sort and summarise your notes, so “what will actually run here” is the first question every install has to answer. The failure is unhelpful: a model that does not fit does not refuse to load, it just spills to system RAM and gets ten times slower. Varven checks this for you at install time — this is the same arithmetic, made public.
See what Varven does · Read the docs
Common questions
How much VRAM do I need to run a 7B model?
At Q4_K_M, roughly 4.4 GB for weights plus KV cache and about 0.8 GB overhead. An 8 GB card handles it comfortably at 8k context. At FP16 the same model wants about 14 GB and will not fit.
What happens if a model does not fit?
Layers spill to system RAM and run on the CPU. It still works, but throughput often drops by more than ten times. For anything interactive, pick a smaller model or a heavier quantisation that fits entirely in VRAM.
Does context length matter?
Yes, and more than people expect. KV cache grows linearly with context, so doubling from 8k to 16k doubles it. A model that loads fine can still run out of memory deep into a long conversation.
Which quantisation should I pick?
Q4_K_M is the sensible default. Q5_K_M and Q6_K are better if you have room. Below Q3 the quality loss becomes obvious on anything requiring reasoning.