Ollama on a CPU-only VPS: tokens per second by plan

Can a VPS without a GPU run a language model fast enough to be useful? We stopped guessing and measured it: eight models with Ollama, on each of our four plans, same prompt, same settings. This page has the numbers, what they mean in practice, and which plan to pick for which model.

1. How we measured

One VM ordered like yours, Ubuntu 24.04 from our image, resized to each plan in turn: Nano (1 vCPU, 2 GB), Starter (2 vCPU, 4 GB), Pro (4 vCPU, 8 GB) and Scale (8 vCPU, 16 GB). The processor seen by the VM is an Intel Xeon Gold 6252 with AVX-512, which llama.cpp, the engine inside Ollama, uses. Ollama 0.40.3, default quantization of each model (Q4), no GPU anywhere.

For each model: a one-paragraph question, 128 tokens of answer, temperature 0. One warm-up request loads the model, then two measured requests; the figures below are their average. The two runs never differed by more than 12 %. Each request starts differently, so Ollama cannot reuse the previous prompt from its cache.

2. Generation speed, in tokens per second

"In memory" is what ollama ps reports once the model is loaded with a 2,048-token context. "Does not fit" means exactly that: the model plus the system did not fit in the plan's RAM. Ollama does not slow down in that case. Either it notices in time and refuses, or the kernel stops it and the request fails:

llama-server process has terminated: exit status 1: ggml_aligned_malloc: insufficient memory
Remote end closed connection without response

Count about 1 GB for the system and Ollama itself on top of the model, more if you raise the context length.

3. More cores help less than you would think

Going from 2 to 4 vCPU speeds generation up by 1.5 to 1.8 times. Going from 4 to 8 only adds 13 % on the smallest models and 34 to 60 % on the 3B to 8B ones. Writing each token means reading the whole model from memory, so beyond a few cores the memory bandwidth sets the pace, not the processor.

Reading your prompt is different: the model processes all its tokens at once, and that work does scale with cores. On the 8B models, the Scale reads the prompt 1.8 times faster than the Pro:

This matters as soon as the input is long. Asking an 8B model on the Pro to summarise a 2,000-token document means about 85 seconds before the first word of the answer; on the Scale, about 45.

4. What these speeds feel like

Most people read around 240 words a minute, which is roughly 5 tokens per second. That gives a simple scale:

• Above 10 tokens/s: the answer appears faster than you read it. A chat feels live.
• 5 to 10 tokens/s: you read as it writes. Fine for an assistant you use now and then.
• Below 5 tokens/s: you wait. Use it for jobs nobody watches: classifying emails overnight, extracting fields from documents, a bot that answers within the minute.

The first request after a pause also pays for loading the model from disk: 7 seconds for qwen3:0.6b, 33 for hermes3:8b, 98 for qwen3:14b on the Scale. Ollama unloads a model after five idle minutes by default; OLLAMA_KEEP_ALIVE keeps it in memory longer, as described in our Hermes and Ollama guide.

5. Which plan for which model

Billed by the hour, a Scale costs 0.116 CAD an hour: enough to test a model for an afternoon before deciding. Beyond 14B, or for many users at once, a CPU is no longer the right tool.

6. Measure it on your own machine

Ollama returns its own timings. The generation speed is eval_count divided by eval_duration, which is in nanoseconds:

curl -s http://127.0.0.1:11434/api/generate \
  -d '{"model": "qwen3:4b", "prompt": "Explain what a VPS is.", "stream": false, "think": false,
       "options": {"num_predict": 128, "temperature": 0}}' |
  jq '{tokens: .eval_count, tokens_per_s: (.eval_count / (.eval_duration / 1e9)),
       prompt_tokens_per_s: (.prompt_eval_count / (.prompt_eval_duration / 1e9))}'

Run it twice and keep the second result: the first one includes loading the model.