AI Workstation Calculator
Score your build for AI work. See which LLM sizes you can run at 7B, 13B, 34B and 70B, plus Stable Diffusion and model training capability.
What matters for local AI work
One number dominates everything else: VRAM capacity. A model that fits in video memory runs at full speed. A model that does not either spills into system memory — often ten times slower — or refuses to load at all. Compute throughput only becomes the limiting factor once the model fits.
Working out what fits
Multiply the parameter count by the bytes per parameter at your chosen precision, then add roughly 20% for the key-value cache and activations:
VRAM (GB) ≈ parameters (billions) × bytes per parameter × 1.2
A 7 billion-parameter model at FP16 needs about 17 GB; the same model at INT4 needs about 4 GB. That is why quantisation, not a bigger budget, is usually the right answer.
Quantisation costs less quality than people expect
Reducing precision from 16-bit to 4-bit sounds drastic, but modern quantisation methods preserve output quality remarkably well — the difference is often barely measurable on general tasks. Running a larger model at INT4 almost always beats running a smaller one at FP16.
System memory still matters
Models are read into system RAM before being transferred to the graphics card, and offloaded layers live there when a model does not fully fit. 32 GB is a sensible floor and 64 GB gives room to work with larger models.
CUDA remains the path of least resistance
Most AI tooling targets NVIDIA first. AMD's ROCm and Intel's equivalent have improved substantially, but you will meet more projects that assume CUDA, and more guides written for it. That is a practical consideration rather than a performance one.
Fine-tuning beats training
Training a model from scratch on a single consumer card is not realistic. Fine-tuning an existing model with LoRA or QLoRA is, and it is where consumer hardware genuinely earns its place — a 24 GB card handles meaningful fine-tuning work.
Frequently asked questions
How much VRAM do I need to run a 7B model?
About 17 GB at FP16, 9 GB at INT8, or roughly 5 GB at INT4. Quantisation is what makes these models practical on consumer graphics cards.
Is 12GB of VRAM enough for local AI?
It comfortably runs 7B and 13B models quantised, and Stable Diffusion at reasonable resolutions. It is not enough for 30B-class models or for serious fine-tuning.
Does quantisation hurt model quality?
Less than most people expect. INT8 is close to indistinguishable, and INT4 loses a small amount on complex reasoning. A larger model at INT4 almost always beats a smaller one at full precision.
Can I use an AMD card for AI work?
Yes, and ROCm has improved considerably. You will meet more projects that assume CUDA and more setup friction, so factor in your own patience alongside the specifications.
Do I need a powerful CPU for AI?
Not for GPU inference, where the processor mostly feeds data. It matters for data preprocessing, for classical machine learning, and when layers are offloaded to system memory because the model does not fit.