Kayon documentation
Everything you need to turn a Windows + NVIDIA machine into an honest, private, local LLM workstation. Start here, then jump to whatever you're stuck on.
Kayon does one thing well: it runs open-weight language models on hardware you already own, and it tells you the truth about what will actually run before you spend an evening downloading it. There is no account, no cloud dependency, and no telemetry unless you turn it on.
What's in this guide
- Getting started walks you from download to your first local chat.
- Core concepts explains fit verdicts and the memory model that powers them, plus how downloads and Ollama adoption work.
- Using Kayon covers day-to-day chat, the tools a capable model can call, the files and artifacts a session produces, and the privacy surface.
- Reference has system requirements and fixes for common snags.
New to local models?
You don't need to understand quantization or KV caches to use Kayon. It computes a plain-language verdict for every model. Read Fit verdicts and you're set.
Install & first run
From a fresh download to a probed machine in under two minutes.
1. Download and open
Grab the installer for Windows and run it. Kayon ships as a small desktop app with a managed llama.cpp runtime bundled in, so there is nothing else to install and no Python environment to manage. It installs to your user account, so it needs no administrator prompt.
Upgrading from 1.4.1 or earlier?
Versions up to 1.4.1 installed machine-wide, so this one will not replace them. Uninstall the old Kayon first, under Settings, Apps, Kayon, then run this installer. Your models, chats and settings live in your profile and survive either way.
2. Let it probe your hardware
On first launch Kayon reads your machine directly through NVML: GPU model, VRAM, driver and CUDA version, CPU, system RAM, and free disk per drive. Nothing is guessed and nothing is uploaded. The probe runs locally and repeats live at 1 Hz while the app is open.
3. Meet your best pick
First run ends on a computed recommendation: the single model and quant that runs best on your GPU, with the honest verdict beside it. It is calculated from your hardware, never a hardcoded default. Install it, or skip and explore with zero models. Everything in Kayon works before you download anything.
No supported GPU?
Kayon still runs. Without a discrete NVIDIA card it computes verdicts against system RAM and CPU inference instead, so you always know what's possible.
4. Staying up to date
Kayon checks on launch whether a newer version exists and tells you. It does not download anything on its own. Click Download when you want it, and when the download finishes, Relaunch to update applies it. No administrator prompt, because the install belongs to your account.
The check asks GitHub for a version file and sends no identifiers, no telemetry, and nothing about you or your machine beyond the request itself. It is written to the network log like every other request: an update path that skipped the log would make the log a lie. Settings has a switch to turn it off entirely, which stops the request being made at all, and leaves a Check for updates button for when you want one.
Every update is signed with a key that never leaves the developer's machine, and Kayon verifies that signature against a key built into the app before running a single byte. An installer that fails the check is refused, not executed. An updater is the one feature that downloads and runs code, so it is held to a higher bar than the convenience deserves.
This update signature is not the same thing as code signing. Kayon is not code-signed yet, so Windows still warns that the publisher is unknown on first run.
The dashboard
Your machine, measured, all on one screen.
The dashboard is an instrument cluster for your hardware. It reads straight from NVML rather than estimating, so what you see is what the GPU is actually doing.
- GPU identity — model, architecture, driver and CUDA version, total VRAM.
- VRAM ring — dedicated memory split into used, reserved display headroom, and free.
- Live meters — GPU utilization, temperature, power draw, clocks, CPU and system RAM, updated once a second.
- Speed benchmark — a warm-run generation and prompt-eval figure for the loaded model.
- Per-drive disk — free space on each volume, so you know where a model can land.
Fit verdicts
The heart of Kayon. Every quant of every model carries one honest verdict for your exact machine.
A verdict answers the only question that matters before a download: will this actually run, and how well? There are five, plus one honest refusal.
Change the context length or toggle the q8_0 KV cache and every verdict recomputes on the spot. The computed best pick for your GPU always leads the browser.
The memory model
Why Kayon doesn't just check file_size < VRAM.
Most tools decide whether a model fits by comparing its file size to your VRAM. That's wrong often enough to waste your bandwidth and your evening, because a running model needs far more than its weights. Kayon adds up the real costs:
The KV cache scales with your chosen context length and is read GQA-aware from the model's real key_length, not a naive division that guesses low. And the comparison is against dedicated free VRAM, so a model can't quietly spill into shared system memory and crawl while a green light says it's fine.
The last term is the runtime's own working memory, and Kayon does not assume it. That figure depends on the exact llama.cpp build in the box: which buffers it keeps in VRAM, and which it moves to system RAM. Kayon measures the reserve for the build it ships and grows it with your context length, so the number tracks the runtime you actually run rather than one copied from somewhere else.
Why this is measured, not assumed
Kayon pins the exact llama.cpp build it bundles, checksums it, and reserves what that build really uses — checked against what it allocates across several models and context lengths, down to a rounding error. This reserve has been wrong before, in both directions, precisely because it was a constant carried over from a different build. A verdict is only as honest as the numbers behind it, so those numbers are measured against the runtime in the box, not inherited.
Open any verdict
Click a quant in the browser to expand its breakdown: weights + KV + buffers = total need, measured against available VRAM. The math is always visible.
Downloading models
Resumable, checksummed, and safe by default.
Kayon downloads from a signed, verified catalog. Before a byte lands it runs a disk pre-flight to confirm the target volume has room. Downloads are resumable, so a dropped connection doesn't cost you the whole file.
- SHA-256 verified — every file is checked against the catalog's pinned checksum on completion.
- Quarantine on mismatch — a file that fails verification is set aside and never enters your library.
- You pick the drive — point the library at any volume with space; the dashboard shows you which qualify.
Adopting from Ollama
Bring the models you already have. Re-download nothing.
If you already keep models in Ollama, Kayon finds them and adopts them in place using NTFS hard links. Zero bytes are copied and nothing re-downloads, as long as your library and the Ollama store sit on the same NTFS volume.
A hard link means both Kayon and Ollama point at the same blob on disk. Delete Kayon's link later and Ollama's copy stays exactly where it was. Adopted models show an adopted · ollama badge in your library and get the same honest verdict as everything else.
Different volume?
Hard links only work within one NTFS volume. If Ollama lives on another drive, Kayon offers a copy instead and tells you the cost up front.
Chat & inference
Talk to a model that runs entirely on your GPU.
Load any model from your library and chat. Kayon runs it through the managed llama.cpp server on 127.0.0.1, so your prompts never leave the machine. You get streaming output, reasoning segments where the model supports them, and, on models whose chat template supports it, tool calling.
- Live inference stats — generation and prompt-eval tokens per second, VRAM in use, GPU utilization, and context consumed, from the same telemetry as the dashboard.
- Sampling controls — system prompt, temperature, top-p, and max tokens in a panel you can collapse for a wider reading column.
- Local history — conversations stay on your disk. There is no sync and no server that sees them.
A capable model can also act, not just answer. See Tools for what it can reach and what it must ask you first. Files you attach, and files the model writes, collect in the session workspace and are listed under Files & artifacts.
Tools
A capable model can act: on files in a scoped workspace, and on the web only if you let it.
When a loaded model supports tool calling, Kayon offers it a built-in tool set and runs an agent loop. The model asks for a tool, Kayon runs it locally, the result goes back, and the loop repeats up to a bounded number of iterations until the model has a final answer. Every step appears in the chat and is saved with it.
Which models get tools
Kayon decides at load time by reading the model's GGUF chat template for tool-call markers. There is no hand-curated list of tool-capable models and nothing is guessed: if the template does not support tools, Kayon does not offer them.
What a turn can actually reach is the intersection of three things: the model supports tools, the session has a workspace, and the Web toggle is on. Switch the Web toggle off and the network tools are not in the set the model is offered at all.
The built-in set
- calculator — deterministic math, with no
eval. - read_file, list_dir, write_file — scoped to the session workspace.
- read_selection — reads what you highlighted.
- code — a Python interpreter.
- search, fetch_url — absent unless the Web toggle is on.
read_file extracts text from PDFs, so an attached document can be read and summarized. It refuses non-PDF binaries with a clear message instead of returning garbage, and it forgives a truncated filename by resolving it to a unique file in the workspace. On a real miss it lists the actual names, so the model retries instead of inventing an answer.
The set is built in and Kayon-vetted. MCP and user-defined tool servers are a documented post-v1 extension.
The session workspace
Every session has a workspace, and the filesystem tools and the code interpreter work only inside it. Either you attach a folder, or Kayon creates one for that chat at ~/.kayon/workspace/<session>/. Files you attach are copied in so the model can read them, and files the model writes land there as artifacts.
The resolved real path has to stay inside the workspace. .., absolute paths, and symlinks pointing out of it are refused. The blast radius is the workspace.
What deleting a chat deletes
Deleting a chat deletes the workspace Kayon created for it, and the artifacts and attached copies inside it. A folder you attached is never removed or emptied: it is your directory, possibly a source tree, and Kayon is a guest in it. Kayon deletes only what Kayon created.
Web access is opt-in
The Web toggle lives in the chat and is off by default, including on sessions where you attached a folder. It gates search and fetch_url; nothing else in the tool set touches the network.
The default provider is DuckDuckGo, which needs no API key, no account, and no configuration. Queries go straight from your machine to the provider, so Kayon is never a proxy and never sees what you searched for. You can bring your own keyed provider in Settings if you want the reliability. Every query and every URL goes through the same instrumented client as everything else and shows up in the network log.
Confirmation is the boundary
Read-only tools just run: calculator, read_file, list_dir, search. The side-effectful ones are gated.
- code — always asks, on every call, whatever folder you are in.
- write_file — asks when writing into a folder you attached. Writes into Kayon's own workspace flow through, so model-made artifacts appear without a click.
A per-session auto-approve exists for a workspace you trust. It is off by default and resets whenever the workspace changes.
Code execution is not a sandbox in v1
Approved code runs as a subprocess with your OS permissions. Its working directory is the workspace, which is a sensible blast-radius default, and it runs in isolated mode and is killed on timeout. None of that is a security boundary: code you approve can reach outside the folder. The confirmation is the boundary. A real WASM or OS jail is documented post-v1 hardening.
This is the opposite of the artifact viewer, where the frame is a real, browser-enforced boundary. The two look similar and are not: a sandboxed frame earns automatic rendering, a subprocess holding your permissions does not.
Nothing is hidden
Every tool call shows inline with its name, its arguments, and its result, and it is saved with the conversation. Tool errors are surfaced rather than swallowed. A call waiting on your approval stays expanded, because its arguments are the thing you are approving.
Files & artifacts
Open a document without leaving Kayon, and without it leaving your machine.
The Files panel in Chat lists the session workspace: the documents you attached and the files the model wrote, together in one place. Click any of them to open it inline.
- Markdown, text, and code — rendered with highlighting.
- Images — shown inline.
- PDFs — real pages with paging and zoom, not extracted text.
- HTML — rendered as markup. See the artifact note below.
The PDF engine and every asset it needs are bundled into the installer, so opening a document never fetches anything and never sends it anywhere. It costs a couple of megabytes of download, and it is the reason a policy document you open here cannot leak.
The viewer is read-only. Saving a copy is an explicit click; nothing is written to your disk for you.
Artifacts render, but they do not run
An artifact's JavaScript does not execute. This is deliberate. A content policy can stop a page fetching, but nothing stops a script navigating itself to https://somewhere/?your=data — and a navigation is not a fetch, so no policy catches it. This was tested rather than assumed: with scripts enabled, a test artifact really did reach a live external site.
Running artifact scripts and promising that nothing leaves your machine cannot both be true. The promise wins. So a chart or React artifact shows as static markup, the viewer says so on the artifact itself, and it names the remote URLs it refused to load. Save a copy to run it in a browser you trust.
Privacy & network
Private by construction, not by policy.
Every outbound request in Kayon passes through one instrumented client and lands in a log you can read. Nothing else can open a socket. There is no account to create and no identity for a server to know.
Four things can leave, and you can see all of them in that log: model downloads and catalog lookups from Hugging Face, the web tools if you switch them on, telemetry if you switch it on, and the update check. The update check carries no identifiers and can be switched off. Nothing else opens a connection.
Telemetry is opt-in, and honest
Telemetry stays off until you switch it on. When you do, Kayon shows you the literal payload — the exact bytes it would send — and holds it until you confirm. You approve the data, then it sends. Never the other way around.
Local traffic to 127.0.0.1 — the llama-server sidecar — is not egress and never appears in the network log. It never leaves your machine.
Requirements
llama.cpp, Vulkan build. Nothing to install separately. Vulkan is slower than CUDA on NVIDIA, and it is 20 MB against roughly 600 MB, which is the trade.Not in v1: macOS, AMD GPUs, fine-tuning, server mode, and multi-agent orchestration. Kayon would rather do one job honestly than half-support everything.
Troubleshooting
A model was marked green but runs slowly
Check the loaded context length. A verdict computed at 4K can loosen or tighten as you raise it, because the KV cache grows with context. Kayon recomputes when you change it in the browser; make sure the value you're chatting at matches.
My Ollama models won't adopt
Adoption uses NTFS hard links, which only work within a single volume. If your library and the Ollama store are on different drives, move one or let Kayon copy instead.
A download failed verification
The file didn't match the catalog's pinned SHA-256 and was quarantined so it can't be loaded. Retry the download; resumable transfers pick up where they left off.
I see "needs newer runtime"
The model's architecture is newer than the bundled llama.cpp build. Update Kayon to pick up a newer signature-gated runtime, then the model becomes loadable.