A model at home on your PC

This page is the hands-on companion to Own model in the cloud: how to run a model on your PC with a graphics card so that the killBottleneck assistant reaches it from anywhere — and nobody else does. No programming, three commands. Allow an hour including the model download.

What works at the end

A model runs on the PC, listens only to itself (on the local address) and wants a key. A tunnel gives it a public HTTPS address. You enter that address, the model name and the key in killBottleneck — and the assistant thinks at home even when you are at the office or on your phone in the car. Turns do not count towards the quota, data never leaves your house.

What you need

• A graphics card with 16 GB of memory (e.g. NVIDIA RTX 4060 Ti/5060 Ti 16 GB, RTX 3090/4090) for gpt-oss-20b, the model we have verified with the assistant. With 24 GB, Qwen3.8 (27B) works too. 8 GB cards run only smaller models, and those call the assistant's tools unreliably.
• Windows 10/11, Linux or Mac and about 15 GB of free disk space.
• An ordinary internet connection. The assistant sends short texts and gets short replies, line speed does not matter; but the PC must be on whenever you use the assistant.

Step 1 — install llama.cpp

llama.cpp runs the model and serves it in the OpenAI format. You only need llama-server from it.

• Windows: in Releases download the latest …-bin-win-cuda-x64.zip (NVIDIA; plus cudart-… from the same release) or …-bin-win-vulkan-x64.zip (AMD, Intel) and unpack it, e.g. into C:\llama.
• Mac: brew install llama.cpp.
• Linux: the package from Releases (…-bin-ubuntu-x64.zip) or build it per the project README.

Step 2 — make up a key

The key is the password killBottleneck shows to the model. Without it anyone who guesses the address could compute on your graphics card. Make a long random one:

# Windows (PowerShell)
-join ((48..57)+(65..90)+(97..122) | Get-Random -Count 32 | ForEach-Object {[char]$_})

# Mac / Linux
openssl rand -hex 16

Store it in your password manager — you will enter it in two places (the model and killBottleneck) and it belongs nowhere else.

Step 3 — start the model with the key

In a terminal (Windows: PowerShell in the folder with llama-server.exe):

llama-server -hf ggml-org/gpt-oss-20b-GGUF --alias gpt-oss-20b \
  --host 127.0.0.1 --port 8080 --api-key MY-SECRET-KEY -c 16384 -ngl 99

What the parts mean:

part	why
-hf ggml-org/gpt-oss-20b-GGUF	the model downloads itself on first start (~12 GB, takes a while)
--alias gpt-oss-20b	the name killBottleneck sees the model under — this goes into the Model field
--host 127.0.0.1	listens only to itself — nobody on the network or the internet sees it
--api-key MY-SECRET-KEY	the key from step 2
-c 16384	conversation memory; the assistant sends map extracts too, do not go below 16 thousand
-ngl 99	whole model on the graphics card (otherwise the CPU computes and replies take minutes)

killbottleneck.com

A correctly started model: 25/25 layers on the GPU, listening on 127.0.0.1 only.

Once the line server is listening on http://127.0.0.1:8080 appears, open http://127.0.0.1:8080 in a browser. You will see the llama.cpp interface asking for the key — that is right, it must behave the same towards anyone else.

Step 4 — give the model a public address (tunnel)

Never open port 8080 on your router

"Port forwarding" exposes the model to the whole internet without encryption and without the router's protection. A tunnel does the opposite: only an outgoing connection leaves your PC, no port opens, HTTPS is free and only that one program is visible.

A) Cloudflare Tunnel — to try it out (the address changes on every start)

Install cloudflared (Windows winget install Cloudflare.cloudflared, Mac brew install cloudflared, Linux package per the Cloudflare guide) and in a second terminal window run:

cloudflared tunnel --url http://127.0.0.1:8080

killbottleneck.com

The address in the box is what you enter in killBottleneck (plus /v1).

B) Tailscale Funnel — permanent, free (the address stays the same)

If you use Tailscale (free for home use), all it takes is:

tailscale funnel --bg 8080

You get a fixed address like https://pc-name.your-tailnet.ts.net. On first run Tailscale sends you to a page where you allow Funnel for this computer. Note: the tailnet address itself (100.x.y.z) does not work from the cloud — it is private; Funnel does.

C) Your own domain, permanently — create a named tunnel in Cloudflare (cloudflared tunnel create …, guide at Cloudflare); the address is then e.g. https://gpu.your-domain.com and survives restarts.

Step 5 — enter it in killBottleneck

Organization settings → Own model or API key:

field	what to enter
API address	the tunnel address + /v1, e.g. https://quiet-maple-garden-example.trycloudflare.com/v1
Model	gpt-oss-20b (what you gave --alias)
API key	the key from step 2

Click Test connection — within seconds you know whether it fits. Then turn on the switch and Save. The whole screen and the meaning of the fields are described in Own model in the cloud.

Advanced: gpt-oss can "think", which delays the reply. Put {"chat_template_kwargs":{"reasoning_effort":"low"}} into the extra fields and the assistant answers faster.

Ten safety rules

1. The model listens only on 127.0.0.1 — never 0.0.0.0.
2. Always a key (--api-key), long and random; the same key nowhere else.
3. No open port on the router. Only a tunnel.
4. The tunnel exposes only port 8080 — not your desktop, files or other programs.
5. Changing the key? Change it in both places at once (the model and killBottleneck).
6. Not using the model for days? Stop the tunnel (Ctrl+C) — what is not running cannot be attacked.
7. Update llama.cpp like any program (new release = new zip).
8. Do not send the tunnel address anywhere — whoever knows it and guesses the key computes on your card.
9. Did a quick-tunnel address end up in the wrong hands? No harm: after a restart it is different.
10. What leaves killBottleneck is the conversation text and map extracts — to your home. Photos and voice notes stay on our AI (see Where the data goes).

Making it run by itself

• Turn off sleep. Windows: Settings → Power → Sleep: Never (the screen may turn off). A sleeping PC = the assistant says "the model is not responding".
• Start on boot. The simplest is a start-model.cmd batch with two lines (llama-server … and cloudflared …) in Task Scheduler "At log on"; on Linux two systemd services (Restart=always).
• A fixed address. The Cloudflare quick tunnel changes its address on every start — for permanent use pick Tailscale Funnel or a named tunnel, otherwise you keep retyping the address in killBottleneck.
• Power. The graphics card draws a few watts idle; while answering, for a few seconds, as much as when gaming. At home that is pennies a day — which is why we charge nothing for turns on your own model.

When it does not work

what you see	where to look
Test: cannot connect	the tunnel is not running, or /v1 is missing at the end of the address
Test: 401 / unauthorized	the key in killBottleneck differs from --api-key
Test: model not found	the name in the Model field ≠ --alias
assistant: the model is not responding	the PC fell asleep, the tunnel dropped, or the reply takes over 100 s — try reasoning_effort: low or a smaller model
replies take minutes	the model is not on the GPU — -ngl 99 missing, or it did not fit into the card's memory
the assistant only talks, does nothing	the model cannot call tools — go back to gpt-oss-20b

Models that have proven themselves

model	GPU memory	note
gpt-oss-20b (ggml-org/gpt-oss-20b-GGUF)	16 GB	verified with the assistant, calls tools reliably
Qwen3.8 27B	24 GB	verified, slower start
models up to 8 GB	8 GB	they answer, but create steps and My day items through tools unreliably

If you want a provider (OpenAI, OpenRouter…) instead of a home PC, you need no tunnel — just their address and key, see Own model in the cloud.