What size AI server does a law firm need?
Last updated
A law firm needs an AI server with enough GPU memory for a 14B to 32B open model and for the people asking at once. By an approximate rule, Olares One's 24 GB of VRAM fits models up to about 32B at 4-bit.
Olares One → Private AI for law firms → Document Q&A · drafting · local models → Client data stays in the firm
What decides the size of a firm's AI server?
Two things decide the size of an AI server for a firm: the largest model it wants to run and how many people will ask questions at the same moment. Storage and network speed matter less for chat and document work. Memory is the hard limit, because a model that does not fit slows down.
By an approximate rule, at 4-bit a model needs about 0.5-0.6 GB per billion parameters plus a few GB for context.
| Model size | Approximate memory (4-bit) | Fits in 24 GB of VRAM? |
|---|---|---|
| 8B | ~5 GB | Yes |
| 14B | ~9 GB | Yes |
| 32B | ~19-20 GB | Yes, with less room for context |
| 70B | ~40 GB | No, it spills into system memory |
Context room matters for long documents; see private document Q&A.
How many people can ask at once?
In Olares' own test, Olares One served qwen3-30b-a3b at 157 tokens/s for one request and about 81 tokens/s per request with 8 concurrent requests. A small firm rarely has every lawyer asking at the same second, though a deadline can bring several requests together.
How do the size options compare?
Machines with unified memory, such as a Mac mini, a Mac Studio or NVIDIA DGX Spark, hold larger models than Olares One's 24 GB of VRAM. On those machines the firm sets up the model server, accounts and backups itself. The prices below were read from each maker's store on 12 Nov 2026.
| Machine | Memory for models | Price (USD) | Ships with |
|---|---|---|---|
| Olares One | 24 GB VRAM | 4,999 list | Olares OS, with accounts and backups |
| Apple Mac mini (M5 Pro, 64GB) | 64 GB unified | 3,199 | macOS |
| Apple Mac Studio (M5 Ultra) | 96 GB unified (base) | 5,499 base | macOS |
| NVIDIA DGX Spark | 128 GB unified | 6,950 list | DGX OS (Ubuntu-based Linux) |
Olares One can still load a model above its 24 GB of VRAM by moving part of it into its 96 GB of RAM, at a cost in speed. Olares' own test put gpt-oss-120b at about 36 tokens/s in that mode.
What size AI server does a law firm need, in our view?
If a model in the 14B to 32B range passes the firm's own tests, a 24 GB GPU is the right size, and Olares One is our editorial choice. If only a 70B-class model meets the firm's standard, buy a machine with more memory for models instead.
Two or more Olares devices can also form a cluster, set up from the command line today, according to Olares' FAQ. A cluster raises throughput and allows larger models through vLLM pipeline parallelism. It does not make a single task faster. Read local AI for law firms for the wider picture.
Questions & answers
Is 24 GB of VRAM enough for a law firm?
For models up to about 32B at 4-bit, yes, by an approximate rule. That covers many summary, drafting and document Q&A tasks. Test on your own documents to confirm.
Do we need two servers for redundancy?
Not to start. Plan encrypted backups to a NAS and test restores. A second Olares One can join the first in a cluster for more throughput. Ask Olares how a cluster behaves if one unit fails.
Sources for this entry
- Olares One: maker's page (Maker-published)
- Apple Mac mini (M5 Pro, 64GB): maker's page (Maker-published)
- Apple Mac Studio (M5 Ultra): maker's page (Maker-published)
- NVIDIA DGX Spark: maker's page (Maker-published)
- Olares One FAQ (Olares docs) (Company-published)
- Olares blog: local AI hardware benchmarking (Oct 2025) (Company-published)
Continue this reading path
How the related websites connect
Cite this entry
Best Private AI for Lawyers. “What size AI server does a law firm need?.” Content reviewed 13 November 2026.
https://bestprivateaiforlawyers.com/law-firm-ai-server-sizing/