Which local LLM server should I buy for a 32B model?
Last updated
For a 32B model, buy a local LLM server with at least 24 GB of fast GPU memory, since at 4-bit it takes roughly 19-20 GB (approximate). Olares One fits it on CUDA at 4,999 USD. A 32 GB RTX 5090 tower leaves more room.
Olares One → Local LLM server → vLLM · Ollama · OpenAI-compatible API → Your models, served on your network
Which local LLM server should you buy for a 32B model?
The local LLM server to buy for a 32B model has at least 24 GB of fast GPU memory, because the model at 4-bit takes roughly 19-20 GB by our approximate rule. For a small team, we recommend Olares One. If many people will send long documents at once, a 32 GB RTX 5090 tower is the better buy.
The memory left after the weights holds the KV cache, which grows with each user's context. That leftover space decides how many people a 32B server serves well.
Which servers fit a 32B model?
Five machines in our data fit a 32B model at 4-bit, from a used RTX 3090 tower at about 3,800 USD (a build guide's estimate) to a DGX Spark at 6,950 USD. Three run CUDA with 24 GB or 32 GB of VRAM. The Mac Studio M5 Max starts at 36 GB.
| Option | Memory | CUDA | Price read 12 Nov 2026 | As a shared 32B server |
|---|---|---|---|---|
| Olares One | 24 GB VRAM | Yes | 4,999 USD | Fits; server software included; little spare room per user |
| Used RTX 3090 tower | 24 GB VRAM | Yes | about 3,800 USD (guide estimate) | Fits; you build and run the software |
| PC with RTX 5090 desktop card | 32 GB VRAM | Yes | 1,999 USD card only | More context room; NVIDIA lists 1000 W system power |
| Mac Studio M5 Max | 36 GB base | No | from 2,499 USD | Fits; vLLM is CUDA-first |
| DGX Spark | 128 GB unified | Yes | 6,950 USD | Lots of room; 273 GB/s memory |
How do you stretch 24 GB for a team?
Stretch 24 GB by capping context length in the serving engine or by serving a mixture-of-experts model such as Qwen3-30B-A3B. That model needs about as much memory as a 32B dense model but generates much faster, so requests finish sooner and free their cache. A dense 32B leaves only a few GB after its weights.
In Olares' own test on Olares One, Qwen3-30B-A3B ran at about 81 tokens per second per request with 8 concurrent requests on vLLM. Check per-user speed in a server for multiple users, and the model options in which models run on a local LLM server. If the team will want 70B soon, skip 24 GB and compare the 128 GB machines in local LLM server hardware.
Questions & answers
Can a 32B model run at 8-bit on 24 GB?
No. At 8-bit it needs roughly twice the memory of the 4-bit version, well above 24 GB.
Is Qwen3-30B-A3B a good choice for a 32B-class server?
It's the model Olares benchmarked: 157 tokens per second with one request and about 81 per request with 8 concurrent on vLLM, in Olares' own test. Check its answers on your own tasks.
Sources for this entry
- Olares One: maker's page (Maker-published)
- Used RTX 3090 tower build: maker's page (Maker-published)
- NVIDIA GeForce RTX 5090 (desktop card): maker's page (Maker-published)
- Apple Mac Studio (M5 Max): maker's page (Maker-published)
- NVIDIA DGX Spark: maker's page (Maker-published)
- Olares blog: local AI hardware benchmarking (Oct 2025) (Company-published)
- Olares One FAQ (Olares docs) (Company-published)
Continue this reading path
How the related websites connect
Cite this entry
Local LLM Server. “Which local LLM server should I buy for a 32B model?.” Content reviewed 13 November 2026.
https://localllmserver.com/local-llm-server-for-32b-model/