How to Self-Host Ollama on a VPS: Setup, Security, Performance & Cost

Why Self-Host Ollama?

Cloud AI APIs are convenient, but they also place the model, infrastructure and pricing structure outside your control. With Ollama, the model runs on a server that you manage. That can be useful when privacy, predictable infrastructure costs or deep application integration matters more than having access to the largest commercial models.

Ollama provides a straightforward runtime for downloading and serving popular open models. Instead of manually configuring inference engines, model formats and server processes, you can pull a model with a single command and immediately communicate with it through a local API.

What Do You Need to Self-Host Ollama?

The right server depends heavily on the model you plan to run. A small 3B-class quantized model is much easier to host than a 20B, 32B or 70B model. RAM and VRAM usually matter more than simply buying the VPS with the highest number of CPU cores.

Step 1: Create Your VPS

ssh root@YOUR_SERVER_IP

Step 2: Install Ollama

Once the server is ready, Ollama’s Linux installer makes the initial installation very short.

curl -fsSL https://ollama.com/install.sh | sh

After installation, verify that the service is available:

ollama --version
systemctl status ollama

On a correctly configured installation, Ollama normally listens locally on port 11434. Keeping it local is desirable because we do not want the raw API exposed directly to the public internet.

Step 3: Download Your First AI Model

Ollama manages models using the ollama pull command. For a CPU server, start with a relatively small model before attempting a larger one.

ollama pull llama3.2:3b

ollama pull llama3.1:8b

Then list installed models:

You can start an interactive conversation directly from SSH:

Test the local API

curl http://127.0.0.1:11434/api/generate \
  -d '{
    "model": "llama3.2:3b",
    "prompt": "Explain self-hosted AI in one paragraph.",
    "stream": false
  }'

If you receive a JSON response containing generated text, the core Ollama server is working.

How Much RAM Does an Ollama Server Need?

Model size is one of the biggest factors in server sizing. Quantized models can reduce memory requirements substantially, but you still need room for the model, context cache, Ollama itself and the operating system.

These are general planning ranges rather than hard guarantees. Exact requirements vary by quantization, model architecture, context size and concurrent requests.

Watch out for multiple models staying in memory

If you switch between several models on a small VPS, memory can become exhausted while the previous model remains loaded. For an inexpensive VPS, it is often safer to keep one primary model loaded rather than frequently jumping between multiple large models.

Step 4: Secure the Ollama API

A raw Ollama endpoint should not simply be opened to the internet. A better design keeps Ollama listening locally and places an HTTPS reverse proxy in front of it.

Configure the firewall

ufw allow OpenSSH
ufw allow 80/tcp
ufw allow 443/tcp
ufw --force enable

ufw status

Notice that port 11434 is not opened. Public traffic should reach your proxy over 443, not Ollama directly.

Point a subdomain to your server

Create an A record such as:

Once DNS is resolving correctly, you can configure a reverse proxy. Caddy is convenient because it can automatically provision HTTPS certificates.

Install Caddy

Follow the current official Caddy installation instructions for your Ubuntu version, then edit:

Protect the endpoint with an authorization key

ollama.yourdomain.com {

    @authorized header Authorization "Bearer CHANGE-THIS-TO-A-LONG-SECRET"

    handle @authorized {
        reverse_proxy 127.0.0.1:11434 {
            header_up Host 127.0.0.1:11434
        }
    }

    handle {
        respond "Unauthorized" 401
    }
}

Reload Caddy after validating the configuration.

caddy validate --config /etc/caddy/Caddyfile

systemctl reload caddy

Why the Host header matters

When Ollama sits behind a reverse proxy, requests can occasionally fail because the upstream application receives your public domain as the Host header. Setting the upstream Host to the local Ollama address helps avoid this class of reverse-proxy problem.

A Simpler Alternative: Use an SSH Tunnel

If only you need to use the server, you might not need to expose an HTTPS endpoint at all. An SSH tunnel lets a port on your local computer securely forward to Ollama on the VPS.

ssh -L 11434:127.0.0.1:11434 root@YOUR_SERVER_IP

You can then connect your local application to:

For a single developer, this is one of the easiest ways to keep the Ollama endpoint completely private.

CPU vs GPU: What Kind of Performance Should You Expect?

The largest practical difference between an inexpensive CPU VPS and a GPU server is response speed. CPU inference can be perfectly acceptable for background jobs, but an interactive assistant feels dramatically better when the model is fully accelerated by a modern GPU.

What Happens When Multiple Users Connect?

Single-user tokens per second tells only part of the story. If you plan to connect a SaaS product or team to Ollama, concurrency becomes extremely important.

A CPU system may continue generating roughly the same total number of tokens per second while multiple users divide that limited throughput. In practice, response latency can become frustrating very quickly.

GPUs can support parallel workloads far more effectively, provided there is enough VRAM for the model, context cache and number of parallel request slots.

Check whether the model is fully loaded on the GPU

nvidia-smi

curl http://127.0.0.1:11434/api/ps

If performance suddenly falls far below expectations, verify that the model has not been partially offloaded to system RAM and CPU.

Is Self-Hosting Ollama Cheaper Than Using an AI API?

Not automatically. This is one of the most important points to understand before renting expensive GPU infrastructure.

An API provider runs massive shared infrastructure with very high utilization. Your rented GPU may sit idle for hours, but you still pay for that idle capacity. As a result, self-hosting is not guaranteed to beat commercial APIs on raw cost per generated token.

Self-hosting becomes particularly appealing when you value data ownership, predictable infrastructure spending, custom model choices, internal applications, offline/private processing or freedom from per-request provider restrictions.

Which Ollama VPS Setup Should You Choose?

Option 1: Personal AI automations

Start small. A CPU VPS with 8–16 GB RAM and a 1B–3B model can handle tasks such as text categorization, draft generation, summaries and data extraction where nobody is waiting for tokens to stream onto the screen.

Option 2: Private ChatGPT-style assistant

For interactive conversation, an NVIDIA GPU with roughly 16–24 GB VRAM is a much more comfortable starting point. You will gain far faster token generation and enough headroom for more capable models.

Option 3: AI features inside a SaaS

Start with a GPU that can hold your entire model in VRAM. Benchmark actual concurrent workloads before launch, because the number of registered users is less important than how many requests arrive at the same moment.

Step 5: Connect Ollama to Your Application

After the server is secured, Ollama can become the model layer behind your website, workflow automation, internal business application or AI SaaS.

Test your HTTPS endpoint

curl https://ollama.yourdomain.com/api/generate \
  -H "Authorization: Bearer YOUR_PRIVATE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2:3b",
    "prompt": "Write one sentence about private AI.",
    "stream": false
  }'

OpenAI-compatible endpoint

Ollama also supports OpenAI-compatible API routes, which makes it possible to use software originally designed for an OpenAI-compatible client by changing the base URL.

Python example

from openai import OpenAI

client = OpenAI(
    base_url="https://ollama.yourdomain.com/v1",
    api_key="YOUR_PRIVATE_KEY"
)

response = client.chat.completions.create(
    model="llama3.1:8b",
    messages=[
        {
            "role": "user",
            "content": "Explain why self-hosted AI can be useful."
        }
    ]
)

print(response.choices[0].message.content)

This architecture allows your application to keep its familiar OpenAI-style interface while directing requests to a model running on infrastructure you control.

Optional: Add a Chat Interface With Open WebUI

Ollama itself is primarily a model runtime and API. If you want a browser experience closer to ChatGPT, Open WebUI is a popular self-hosted interface that can connect to Ollama.

With Open WebUI you can add features such as conversation history, account management, model switching and a graphical interface for users who do not want to interact with the command line.

Common Ollama VPS Mistakes to Avoid

1. Exposing port 11434 directly

Keep the raw Ollama API bound locally and control external access using a properly secured reverse proxy or SSH tunnel.

2. Buying too little memory

A model that nearly fills your server’s RAM leaves little room for context, additional processes or another loaded model.

3. Expecting CPU performance to feel like a GPU

CPU inference may be acceptable for asynchronous workflows but can feel painfully slow for an interactive AI assistant.

4. Ignoring concurrency

A server that feels fast with one request may perform very differently when five, ten or twenty users submit requests simultaneously.

5. Using enormous context windows without checking VRAM

Context caching consumes memory. Large context windows combined with parallel requests can produce out-of-memory failures even when the base model fits comfortably.

6. Assuming self-hosting is always cheaper

Calculate utilization. An expensive GPU that spends most of its day idle can cost substantially more per token than an API.

Is Self-Hosting Ollama Worth It?

For the right workload, absolutely. Ollama provides one of the easiest entry points into running open language models on your own infrastructure.

The strongest reasons to self-host are usually control, privacy, predictable infrastructure and the freedom to build around models without depending entirely on an external AI provider.

The tradeoff is that you become responsible for the server. Security, updates, monitoring, performance tuning, backups and capacity planning are now your job.

Frequently Asked Questions

Recommended Resources