The Machine Should Not Own the Mind
The more I build with AI agents, the more convinced I become that the next hard problem is not simply model intelligence. It is continuity. An agent can appear capable inside a single context window, yet still be fragile in the face of restarts, hardware changes, runtime upgrades, cost shifts, or failure recovery. If its active state is trapped inside one process, one machine, or one vendor runtime, then the agent is not truly persistent. It is merely visiting.
PermeantOS began from that frustration. I wanted to know whether an AI agent’s live inference state could be treated as portable infrastructure rather than as an opaque side effect of a specific runtime. Could we capture the state that gives an agent continuity, move it across machines, and resume generation elsewhere without forcing the target runtime to rebuild the entire prefix from scratch?
That question led to PermeantOS, and to USXF: the Unified State Exchange Format.
If you want to inspect the work directly, the project is available on the PermeantOS website, the GitHub repository, and in the technical white paper.
What PermeantOS Is
PermeantOS is an open-source research preview for live AI state migration. It is not trying to replace inference runtimes such as MLX, vLLM, llama.cpp, or TensorRT-LLM. Instead, it sits around them as a portability layer for runtime state.
In its current form, PermeantOS focuses on KV cache migration. That is intentional. The KV cache is where transformer inference keeps the attention state derived from the prompt. It is also one of the most expensive pieces of state to reconstruct, because rebuilding it requires re-prefilling the entire prefix on the target runtime.
If I can move KV state safely and faithfully, I unlock a more general principle: agent execution should be able to move without forgetting.
Why USXF Exists
USXF, the Unified State Exchange Format, is the runtime-neutral envelope that makes migration possible. It records the information needed to interpret and validate migrated state across different engines, including model identity, attention structure, sequence length, dtype, tensor layout, transfer quantisation, block hashes, checksums, and signatures.
A simple tensor dump is not enough. Different runtimes store attention state differently. They may use different layer layouts, block sizes, cache managers, allocation strategies, and prefix-cache metadata. A portable state format therefore needs to carry not only bytes, but meaning.
USXF exists to make that meaning explicit.
The Evidence Run
The proof point that made PermeantOS feel real was a cross-runtime, cross-host migration from a local Apple Silicon MLX source runtime to an AWS NVIDIA vLLM target runtime.
The test used Qwen/Qwen2.5-0.5B-Instruct, a small enough model to run comfortably on local hardware while still exercising the real transformer KV path. The source host was a local Apple Silicon laptop running MLX. The target host was an AWS g4dn.xlarge instance running vLLM 0.23.0.
The migration path was deliberately concrete:
- Extract source KV state from MLX.
- Normalise the state into USXF.
- Transfer it with integrity metadata.
- Allocate compatible target KV blocks inside vLLM.
- Reshape and write the migrated tensors into the target runtime.
- Seed vLLM prefix-cache metadata.
- Validate the migrated cache through hashes, slot probes, and decode behaviour.
The successful run used a 2016-token prefix across 24 layers. Hash validation passed. Slot probes reported maximum key and value differences of 0.0. vLLM accepted 16 seeded prefix-cache blocks. Most importantly, the source continuation, target baseline continuation, and post-migration target continuation matched exactly for the 16-token validation horizon.
That is the key result: the target runtime did not merely receive a blob. It resumed from migrated state with exact decode fidelity for the tested continuation.
The Failure That Helped
One of the more useful moments in the work was an earlier mismatch at a 2032-token prefix. At first glance, it looked like a migration fidelity issue. In practice, it exposed something more interesting: context-window exhaustion.
The model and validation horizon together exceeded the safe continuation budget. Once that was corrected to a 2016-token prefix, the fidelity gap disappeared. The migrated target and baseline target matched exactly.
That distinction matters. It is the difference between a defective state migration mechanism and a test harness asking the model to continue beyond a valid budget. Good infrastructure research depends on finding that line and documenting it honestly.
Why This Matters
Live state portability changes how I can think about AI systems.
A long-running agent should be able to move from a laptop to a cloud GPU when work becomes heavy. It should be able to move back when cost matters. It should be able to survive host failure, shift between specialised accelerators, or pause and resume without losing the expensive intermediate state that gives it continuity.
This is not just an optimisation. It is a step towards treating AI agents as durable computational processes rather than disposable prompt invocations.
Alan Kay famously said, “The best way to predict the future is to invent it.” For AI agents, I think part of that future is clear: the agent’s state should become more important than the box currently executing it.
What It Is Not Yet
PermeantOS is a research preview, not a finished distributed operating system for agents. The current evidence is deliberately narrow and specific: one model family, one MLX source path, one vLLM target path, and a short exact-match continuation horizon.
There are also pragmatic constraints. Some runtime integrations depend on internal behaviour. Python adapters remain necessary where the underlying ML runtimes are Python-native. Broader model coverage, longer validation horizons, stress testing, and cleaner upstream runtime APIs are all still needed.
That honesty is important. The result is significant because it is real, not because it is complete.
The Road Ahead: Agent Memory Graph Migration
KV cache portability is the first layer. The larger ambition is full Agent Memory Graph migration.
That means capturing not just attention state, but the graph of continuity around the agent: conversation turns, tool calls, tool results, generated artefacts, retrieval memory, pending actions, provenance, token spans, unresolved loops, and policy state. In that world, USXF becomes part of a richer migration package where the KV cache is linked to the agent’s semantic and operational memory.
The result would be closer to process migration for AI agents: not just moving computation, but preserving identity, context, and intent.
Why I Open-Sourced It Now
I decided to release PermeantOS early because this problem should be worked on in the open. Runtime portability, state fidelity, and agent continuity are too important to become locked behind vendor-specific assumptions.
PermeantOS is small, imperfect, and early. It is also concrete. There is a working evidence trail, an arXiv-ready paper, benchmark documentation, migration manifests, and a public roadmap. That feels like the right foundation for a serious open-source conversation.
If you are building agent infrastructure, inference runtimes, memory systems, or distributed AI products, I would love you to explore it, criticise it, and help shape the next layer.
The models will keep changing. The machines will keep changing. The agents I build should be able to endure beyond both.