← Guides

Multi-host Ollama: always-on + workstation routing

~5 min read · 2026-08-11 · Homelab guide · No affiliate links

A pattern that shows up in a lot of serious homelabs: one small box that is always on (NAS, mini PC, or modest GPU server) running Ollama with “good enough” models, plus a stronger workstation that is sometimes online with larger models. You want requests to succeed when the workstation is asleep, and to get better quality when it is awake — without rewriting every client.

This guide covers the architecture, a simple health-based router, timeouts, logging for cost/quality trade-offs, and the security mistakes that turn a LAN helper into an open proxy.

Why two hosts beat one heroic machine

• Always-on baseline keeps automation, agents, and “quick ask” tools working 24/7.
• Boost host holds larger models that do not fit (or do not fit well) on the baseline GPU/RAM.
• Power and noise: the workstation can sleep; the baseline sips power.
• Failure isolation: a Windows update reboot should not take down overnight jobs if the baseline is separate.

Single-host setups are simpler. Move to multi-host when uptime and model size start fighting each other.

Roles

Baseline host

Always online. Runs smaller coder and general models. Bound to a stable LAN IP. Firewall allows only trusted subnets. This is the default destination when nothing else is healthy.

Boost host (workstation)

Optional. Larger models, faster tokens when present. May disappear when the machine sleeps or leaves the network. Clients must treat it as best-effort.

Router

A small program or script that, for each task class (code, reasoning, extraction), walks a preference list: try boost models first if the host is up, else fall back to baseline. Health is usually GET /api/tags with a short timeout.

Preference lists, not hard-coded URLs in every app

Keep model preference in one config file so every tool does not invent its own failover. Conceptually:

# conceptual preference list (YAML-style)
code:
  - host: workstation
    model: qwen2.5-coder:14b   # or your larger coder
  - host: baseline
    model: qwen2.5-coder:7b

reasoning:
  - host: workstation
    model: qwen3:27b
  - host: baseline
    model: qwen3:14b

extraction:
  - host: baseline
    model: nuextract:latest

The router resolves host to a base URL, checks health, then calls /api/chat or /api/generate. If the first choice fails (down, timeout, model missing), try the next. Do not infinite-loop: after the list is exhausted, return a clear error.

Health checks that do not lie

• Short connect timeout (1–3s) so a sleeping workstation does not stall every request.
• Confirm the model exists in /api/tags, not only that the port answers.
• Cache health for 10–30 seconds so a burst of jobs does not hammer a waking machine.
• Cold load: first request after idle can take minutes on large models — separate “health” from “first token latency.” A host can be healthy and still slow to load weights.

Binding Ollama on the LAN

By default many installs listen on localhost only. For multi-host routing, Ollama on each host must accept LAN connections. Typical approach: set the host environment so the server listens on 0.0.0.0 (or the LAN interface), then restrict with host firewall rules to your trusted subnet.

• Allow TCP 11434 (or your chosen port) only from router/agent IPs.
• Do not port-forward Ollama to the public Internet.
• If you must expose it remotely, put auth and TLS in front (reverse proxy, VPN, or both).

Logging that makes the setup worth it

Without logs, you will not know whether the expensive workstation is actually helping. Log at least:

• Task class (code / reason / extract)
• Chosen host and model
• Whether this was a fallback
• Latency to first token and total time
• Error class (timeout, connection refused, model not found)

After a week you can answer: “Did boost host save enough quality to justify leaving it on?” and “Which jobs never need the big model?”

Operational tips from real multi-host pain

1. Pin model names in config; floating latest tags make debugging harder across machines.
2. Disk space: large models on both hosts double storage. Baseline can carry a smaller set.
3. GPU memory: concurrent jobs on one host thrash; queue or limit parallelism per host.
4. Clock skew rarely matters for Ollama, but NTP still helps correlate logs.
5. Agent loops: recursive tools can stampede the GPU — rate-limit per caller.

Security: the part people skip

An open Ollama port on a flat LAN is an unauthenticated compute and data-exfil surface. Anyone who can reach it can send prompts that include whatever context your agents attach. Treat it like an internal API:

• Firewall by source IP
• Prefer WireGuard/Tailscale over raw WAN exposure
• Do not put secrets in prompts that hit a shared host you do not trust
• Keep agent tools least-privilege so a jailbreak cannot wipe disks

Minimal router algorithm

for each candidate in preference[task]:
  if not health_ok(candidate.host):   # cached
    continue
  if candidate.model not in tags(candidate.host):
    continue
  try:
    return chat(candidate, messages, timeout=long)
  except TemporaryError:
    continue
raise NoHealthyModel(task)

That is enough for most labs. You can later add weighted load balancing; start with failover and good logs.

Related

If the problem is “service up on LAN but dead on the public tunnel,” see Origin vs tunnel: reading 502 and 530 and the free OriginReach CLI.

Independent notes from Tactical Data Concepts. Not affiliated with Ollama or Cloudflare. Affiliate disclosure: none in this article.