Self-hosted 3B LLM on your homelab

Two Linux hosts, both ideally LXC containers on a Proxmox node so they’re cheap to provision and snapshot — but that’s not mandatory, separate physical machines on your LAN work just as well. Ollama serves the model on the inference host. OpenWebUI runs as a Docker container on a second front-end host and points at the Ollama API over your LAN. Caddy sits in front of OpenWebUI and terminates TLS so you reach a real https:// URL from any device on your network.

┌──────────────────┐       ┌──────────────────────┐       ┌──────────────────┐
│  Your PC / LAN   │       │   Front-end host     │       │  Inference host  │
│  client          │  ──▶  │   Caddy (TLS)        │  ──▶  │  Ollama :11434   │
│  Browser → HTTPS │       │   OpenWebUI (Docker) │       │  (CPU / iGPU)    │
└──────────────────┘       └──────────────────────┘       └──────────────────┘

1. Sizing the two hosts

Again, this is optional — it can be a normal machine, but I’ll move forward with instructions for LXC containers:

LXC vs VM. Use unprivileged LXC containers for both. They share the host kernel, boot in seconds, and Proxmox snapshots are nearly free. For the OpenWebUI host you do want Docker inside the LXC — make sure to enable nesting in the container’s options, or set features: nesting=1 in the conf.

1.1 Network plan

Suggested addresses — adjust to your subnet. The doc uses these names throughout:

2. Inference host (sc-llm-01) — Ollama on Debian

2.1 Provision the LXC

From the Proxmox host (can also be done locally with any virtualisation tool):

pct create 110 local:vztmpl/debian-12-standard_12.x_amd64.tar.zst \
  --hostname sc-llm-01 \
  --cores 6 --memory 10240 --swap 2048 \
  --rootfs local-lvm:32 \
  --net0 name=eth0,bridge=vmbr0,ip=dhcp \
  --features nesting=1 \
  --unprivileged 1 \
  --onboot 1 \
  --start 1

Get a shell and update:

pct enter 110
apt update && apt -y full-upgrade
apt -y install curl ca-certificates htop

2.2 Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

This installs the binary and registers an ollama systemd service running as the ollama user. By default it binds to 127.0.0.1:11434 — we’ll fix that next so the front-end host can reach it.

2.3 Bind Ollama to the LAN

Drop in a systemd override:

systemctl edit ollama.service

Add:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_ORIGINS=*"
Environment="OLLAMA_KEEP_ALIVE=30m"
Environment="OLLAMA_NUM_PARALLEL=2"

Then:

systemctl daemon-reload
systemctl restart ollama
ss -ltnp | grep 11434      # should show *:11434, not 127.0.0.1:11434

2.4 Pull a 3B model

Three good 3B options. Defaults are Q4_K_M — best speed/quality trade-off.

ollama pull qwen2.5:3b
ollama list                # confirm it's there

2.5 Quick test

ollama run qwen2.5:3b "Reply with exactly the word 'ready'."

Exit the REPL with /bye. For an API check from another machine on the LAN:

curl http://sc-llm-01.lan:11434/api/generate -d '{
  "model": "qwen2.5:3b",
  "prompt": "Reply with exactly the word ready.",
  "stream": false
}'

2.6 Firewall — restrict to the front-end host

Don’t leave 11434 open to the whole LAN. Lock it to the front-end host’s IP using nftables (or on your router firewall — this is to prevent any access from the wider internet). Quick local-host nft sketch:

apt -y install nftables
cat > /etc/nftables.conf <<'EOF'
table inet filter {
  chain input {
    type filter hook input priority 0; policy drop;
    iif "lo" accept
    ct state established,related accept
    icmp type echo-request accept
    ip saddr 192.0.2.0/24 tcp dport 22    accept   # admin from your management subnet
    ip saddr 192.0.2.50   tcp dport 11434 accept   # only sc-chat-01
  }
}
EOF
systemctl enable --now nftables
nft list ruleset

3. Front-end host (sc-chat-01) — OpenWebUI in Docker

3.1 Provision the LXC

pct create 111 local:vztmpl/debian-12-standard_12.x_amd64.tar.zst \
  --hostname sc-chat-01 \
  --cores 2 --memory 2048 --swap 1024 \
  --rootfs local-lvm:8 \
  --net0 name=eth0,bridge=vmbr0,ip=dhcp \
  --features nesting=1,keyctl=1 \
  --unprivileged 1 \
  --onboot 1 \
  --start 1

pct enter 111

3.2 Install Docker

apt update && apt -y full-upgrade
apt -y install curl ca-certificates gnupg
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg \
  | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/debian $(. /etc/os-release && echo $VERSION_CODENAME) stable" \
  > /etc/apt/sources.list.d/docker.list
apt update
apt -y install docker-ce docker-ce-cli containerd.io docker-compose-plugin
systemctl enable --now docker
docker run --rm hello-world      # sanity check

3.3 Run OpenWebUI

Create a minimal compose file at /opt/openwebui/compose.yaml:

mkdir -p /opt/openwebui && cd /opt/openwebui
cat > compose.yaml <<'EOF'
services:
  openwebui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: openwebui
    restart: unless-stopped
    # Bind to localhost only — Caddy will publish it via HTTPS
    ports:
      - "127.0.0.1:8080:8080"
    environment:
      - OLLAMA_BASE_URL=http://sc-llm-01.lan:11434
      - WEBUI_AUTH=true
      - ENABLE_SIGNUP=false        # only the first user can register, then locked
      - DEFAULT_USER_ROLE=pending  # new users need admin approval
    volumes:
      - openwebui-data:/app/backend/data
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 5s
      retries: 3
volumes:
  openwebui-data:
EOF
docker compose up -d
docker compose logs -f openwebui     # watch first boot, Ctrl-C when ready

Replace sc-llm-01.lan with the inference host’s actual hostname or IP. If you don’t have local DNS, use the IP directly. The chat history, users, and settings live in the openwebui-data volume — back this up.

4. Caddy reverse proxy with TLS

Caddy is the right tool here: single static binary, one tiny config file, automatic certificate management, and an internal CA built in for LAN-only deployments. Two paths below — pick one.

4.1 Install Caddy

Still on sc-chat-01:

apt -y install debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' \
  | gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' \
  | tee /etc/apt/sources.list.d/caddy-stable.list
apt update
apt -y install caddy

4.2 Path A — Caddy internal CA (no domain needed)

Caddy generates its own root CA, signs a leaf cert for whatever hostname you point at it, and auto-renews. Trade-off: every device that visits the URL must trust Caddy’s root CA once. Best for a personal homelab.

Edit /etc/caddy/Caddyfile:

chat.lan {
    tls internal
    reverse_proxy 127.0.0.1:8080

    encode zstd gzip
    header {
        Strict-Transport-Security "max-age=31536000"
        X-Content-Type-Options    "nosniff"
        Referrer-Policy           "strict-origin-when-cross-origin"
    }
}

systemctl reload caddy
journalctl -u caddy -n 50 --no-pager     # check it issued a cert

Trusting the root CA. Caddy stores its root at:

/var/lib/caddy/.local/share/caddy/pki/authorities/local/root.crt

Copy it off the host and import it on each client:

• macOS: double-click the .crt → Keychain Access → System keychain → set to “Always Trust”.
• iOS: AirDrop the cert, install profile in Settings, then enable in Settings → General → About → Certificate Trust Settings.
• Linux desktop / Firefox: Firefox keeps its own trust store — Preferences → Privacy & Security → View Certificates → Authorities → Import.
• Windows: import into Local Computer → Trusted Root Certification Authorities via certlm.msc. Not my daily driver, so test before relying on it.

4.3 Path B — Real Let’s Encrypt cert via DNS-01 (recommended if you have a domain)

If you own a domain, point a sub-domain at the front-end host’s LAN IP via your local DNS, and use the DNS-01 challenge to get a publicly-trusted cert without ever exposing port 80/443 to the internet. Zero client-side trust setup.

This requires Caddy compiled with a DNS provider plugin. Easiest route — install xcaddy and rebuild:

# install Go (build-time only)
apt -y install golang-go

# install xcaddy
curl -fsSL "https://github.com/caddyserver/xcaddy/releases/latest/download/xcaddy_$(uname -s | tr A-Z a-z)_amd64.tar.gz" \
  | tar -xz -C /usr/local/bin xcaddy

# build Caddy with the Cloudflare DNS plugin (swap module for your provider)
xcaddy build --with github.com/caddy-dns/cloudflare --output /usr/local/bin/caddy-custom

# replace the system caddy binary
systemctl stop caddy
mv /usr/bin/caddy /usr/bin/caddy.orig
ln -s /usr/local/bin/caddy-custom /usr/bin/caddy
systemctl start caddy
caddy version       # confirm: should include cloudflare in the build info

Caddyfile (using a Cloudflare API token scoped to Zone: DNS edit on the relevant zone):

{
    email [email protected]
}

chat.example.com {
    tls {
        dns cloudflare {env.CF_API_TOKEN}
    }
    reverse_proxy 127.0.0.1:8080
    encode zstd gzip
    header Strict-Transport-Security "max-age=31536000"
}

Put the token in a drop-in env file (mode 600):

mkdir -p /etc/systemd/system/caddy.service.d
cat > /etc/systemd/system/caddy.service.d/override.conf <<'EOF'
[Service]
Environment=CF_API_TOKEN=your-token-here
EOF
chmod 600 /etc/systemd/system/caddy.service.d/override.conf
systemctl daemon-reload
systemctl restart caddy

4.4 DNS — make the name resolve on your LAN

For Path A’s chat.lan, or Path B’s split-horizon entry — three options in increasing order of “proper”:

1. Per-machine /etc/hosts entry. Quickest for a single client.
2. Local DNS rewrite on your router (most consumer firmware supports this).
3. Local zone on your existing resolver — cleanest if you run dnsmasq, Pi-hole, Unbound, or AdGuard Home.

5. First login and OpenWebUI configuration

1. Open https://chat.lan (or your real domain) on any LAN client.
2. Create the admin account — first signup becomes admin. Pick a strong password.
3. In Admin Panel → Settings → Connections, confirm the Ollama endpoint resolves and lists qwen2.5:3b.
4. In Admin Panel → Settings → Users, confirm ENABLE_SIGNUP=false took effect — the signup link should be gone.
5. Pick a model from the top-of-screen selector and chat. First request loads the model into RAM (~5 s on CPU); subsequent requests reuse it for the keep-alive window.

6. Optional — Intel iGPU acceleration (advanced)

Skip this on first deploy. Once everything works on CPU, you can revisit if your inference host has an Intel Arc / Iris Xe / UHD iGPU and you want a free 1.5–2× speedup.

Approach

• Install the Intel compute runtime (intel-opencl-icd, level-zero) in the LXC.
• Pass the iGPU device into the unprivileged container: add dev0: /dev/dri/renderD128,gid=44 to the LXC config.
• Replace the stock Ollama with the ipex-llm-ollama build from intel-analytics/ipex-llm releases — it’s a drop-in replacement.
• Set OLLAMA_NUM_GPU=999 so the runtime offloads as many layers as fit in iGPU shared memory.
• Benchmark with ollama run --verbose qwen2.5:3b 'write a haiku' — compare token/s before and after.

7. Troubleshooting

8. References

• Ollama: ollama.com · github.com/ollama/ollama
• OpenWebUI: docs.openwebui.com
• Caddy: caddyserver.com/docs
• xcaddy + DNS providers: github.com/caddy-dns
• ipex-llm Ollama: github.com/intel-analytics/ipex-llm
• Proxmox LXC docs: pve.proxmox.com/wiki/Linux_Container

Peace!