Kayon documentation
Everything you need to turn a Windows + NVIDIA machine into an honest, private, local LLM workstation. Start here, then jump to whatever you're stuck on.
Kayon does one thing well: it runs open-weight language models on hardware you already own, and it tells you the truth about what will actually run before you spend an evening downloading it. There is no account, no cloud dependency, and no telemetry unless you turn it on.
What's in this guide
- Getting started walks you from download to your first local chat.
- Core concepts explains fit verdicts and the memory model that powers them, plus how downloads and Ollama adoption work.
- Using Kayon covers day-to-day chat and the privacy surface.
- Reference has system requirements and fixes for common snags.
New to local models?
You don't need to understand quantization or KV caches to use Kayon. It computes a plain-language verdict for every model. Read Fit verdicts and you're set.
Install & first run
From a fresh download to a probed machine in under two minutes.
1. Download and open
Grab the installer for Windows and run it. Kayon ships as a small desktop app with a managed llama.cpp runtime bundled in, so there is nothing else to install and no Python environment to manage.
2. Let it probe your hardware
On first launch Kayon reads your machine directly through NVML: GPU model, VRAM, driver and CUDA version, CPU, system RAM, and free disk per drive. Nothing is guessed and nothing is uploaded. The probe runs locally and repeats live at 1 Hz while the app is open.
3. Meet your best pick
First run ends on a computed recommendation: the single model and quant that runs best on your GPU, with the honest verdict beside it. It is calculated from your hardware, never a hardcoded default. Install it, or skip and explore with zero models. Everything in Kayon works before you download anything.
No supported GPU?
Kayon still runs. Without a discrete NVIDIA card it computes verdicts against system RAM and CPU inference instead, so you always know what's possible.
The dashboard
Your machine, measured, all on one screen.
The dashboard is an instrument cluster for your hardware. It reads straight from NVML rather than estimating, so what you see is what the GPU is actually doing.
- GPU identity — model, architecture, driver and CUDA version, total VRAM.
- VRAM ring — dedicated memory split into used, reserved display headroom, and free.
- Live meters — GPU utilization, temperature, power draw, clocks, CPU and system RAM, updated once a second.
- Speed benchmark — a warm-run generation and prompt-eval figure for the loaded model.
- Per-drive disk — free space on each volume, so you know where a model can land.
Fit verdicts
The heart of Kayon. Every quant of every model carries one honest verdict for your exact machine.
A verdict answers the only question that matters before a download: will this actually run, and how well? There are five, plus one honest refusal.
Change the context length or toggle the q8_0 KV cache and every verdict recomputes on the spot. The computed best pick for your GPU always leads the browser.
The memory model
Why Kayon doesn't just check file_size < VRAM.
Most tools decide whether a model fits by comparing its file size to your VRAM. That's wrong often enough to waste your bandwidth and your evening, because a running model needs far more than its weights. Kayon adds up the real costs:
The KV cache scales with your chosen context length and is read GQA-aware from the model's real key_length, not a naive division that guesses low. Compute buffers and CUDA overhead are included. And the comparison is against dedicated free VRAM, so a model can't quietly spill into shared system memory and crawl while a green light says it's fine.
Open any verdict
Click a quant in the browser to expand its breakdown: weights + KV + buffers = total need, measured against available VRAM. The math is always visible.
Downloading models
Resumable, checksummed, and safe by default.
Kayon downloads from a signed, verified catalog. Before a byte lands it runs a disk pre-flight to confirm the target volume has room. Downloads are resumable, so a dropped connection doesn't cost you the whole file.
- SHA-256 verified — every file is checked against the catalog's pinned checksum on completion.
- Quarantine on mismatch — a file that fails verification is set aside and never enters your library.
- You pick the drive — point the library at any volume with space; the dashboard shows you which qualify.
Adopting from Ollama
Bring the models you already have. Re-download nothing.
If you already keep models in Ollama, Kayon finds them and adopts them in place using NTFS hard links. Zero bytes are copied and nothing re-downloads, as long as your library and the Ollama store sit on the same NTFS volume.
A hard link means both Kayon and Ollama point at the same blob on disk. Delete Kayon's link later and Ollama's copy stays exactly where it was. Adopted models show an adopted · ollama badge in your library and get the same honest verdict as everything else.
Different volume?
Hard links only work within one NTFS volume. If Ollama lives on another drive, Kayon offers a copy instead and tells you the cost up front.
Chat & inference
Talk to a model that runs entirely on your GPU.
Load any model from your library and chat. Kayon runs it through the managed llama.cpp server on 127.0.0.1, so your prompts never leave the machine. You get streaming output, reasoning segments where the model supports them, and tool calling for capable models.
- Live inference stats — generation and prompt-eval tokens per second, VRAM in use, GPU utilization, and context consumed, from the same telemetry as the dashboard.
- Sampling controls — system prompt, temperature, top-p, and max tokens in a panel you can collapse for a wider reading column.
- Local history — conversations stay on your disk. There is no sync and no server that sees them.
Privacy & network
Private by construction, not by policy.
Every outbound request in Kayon passes through one instrumented client and lands in a log you can read. Nothing else can open a socket. There is no account to create and no identity for a server to know.
Telemetry is opt-in, and honest
Telemetry stays off until you switch it on. When you do, Kayon shows you the literal payload — the exact bytes it would send — and holds it until you confirm. You approve the data, then it sends. Never the other way around.
Local traffic to 127.0.0.1 — the llama-server sidecar — is not egress and never appears in the network log. It never leaves your machine.
Requirements
llama.cpp with CUDA. Nothing to install separately.Not in v1: macOS, AMD GPUs, fine-tuning, server mode, and multi-agent orchestration. Kayon would rather do one job honestly than half-support everything.
Troubleshooting
A model was marked green but runs slowly
Check the loaded context length. A verdict computed at 4K can loosen or tighten as you raise it, because the KV cache grows with context. Kayon recomputes when you change it in the browser; make sure the value you're chatting at matches.
My Ollama models won't adopt
Adoption uses NTFS hard links, which only work within a single volume. If your library and the Ollama store are on different drives, move one or let Kayon copy instead.
A download failed verification
The file didn't match the catalog's pinned SHA-256 and was quarantined so it can't be loaded. Retry the download; resumable transfers pick up where they left off.
I see "needs newer runtime"
The model's architecture is newer than the bundled llama.cpp build. Update Kayon to pick up a newer signature-gated runtime, then the model becomes loadable.