Agent Harness: A Portable Context System for Local AI Agents
A public architecture note about building a local-first, versioned context system for AI agents with skills, workflows, project briefings, and a compiled LLM Wiki.
Agent Harness: A Portable Context System for Local AI Agents
AI coding agents become useful when they understand more than the current prompt. They need stable context about projects, boundaries, preferences, tools, deployment rules, and prior decisions. Agent Harness is a practical way to move that context out of transient chat history and into a local-first, versioned, portable workspace that multiple agents can read.
It is not a single application, and it is not a cloud memory service. It is a small architecture made of filesystem conventions, a bare Git repository, project briefings, skills, workflows, rules, editor workspaces, and a compiled knowledge layer. The goal is simple: before an agent acts, it should know where to find the right context; while it acts, it should follow explicit boundaries; after it finishes, reusable knowledge should be captured.
The Problem
Many agent workflows are session-centered. During one session, the agent explores a codebase, discovers operational constraints, fixes a deployment issue, and learns several important project facts. In the next session, much of that context is gone or hidden in old transcripts.
Even when a tool has memory, the memory often lacks structure. User preferences, project facts, hard rules, runbooks, temporary notes, and operational procedures can easily blur together. That makes future agents more likely to guess, repeat exploration, or miss a safety constraint.
Agent Harness solves this by separating durable context into layers:
- user profile and communication preferences
- hard rules
- project briefings
- reusable skills
- multi-step workflows
- tool conventions
- compiled knowledge from long-term content
The point is not to load everything every time. The point is to give the agent a reliable map. A project task reads the project briefing. A deployment task reads the deployment workflow. A knowledge task reads the LLM Wiki index.
Core Architecture
A compact Agent Harness can be built around a bare Git repository whose work tree is the user home directory, while tracking only explicit allowlisted files:
~/.agent-harness.git # bare repository
~/.agents/ # portable agent context
~/.claude/CLAUDE.md # user-level dispatcher
~/README.md # harness documentation
~/.gitignore # allowlist and safety rules
This layout has a useful property: the home directory can contain many local tools, caches, logs, secrets, and databases, but Git only sees the few files that are deliberately allowed. The default is to ignore everything. Long-term agent context is then opted in explicitly.
That design keeps the harness portable without turning the entire machine into a dotfiles repository.
Context Layers
The ~/.agents/ directory is the main knowledge root. A practical structure looks like this:
~/.agents/
USER.md
SOUL.md
projects/
skills/
workflows/
rules/
tools/
knowledge/
journal/
workspaces/
USER.md stores factual profile information such as preferred languages, common platforms, and active technical surfaces. SOUL.md stores communication style and collaboration preferences. projects/ contains one briefing per project, usually named OVERVIEW.md. skills/ defines reusable capabilities. workflows/ captures multi-step procedures that span tools or systems. knowledge/ stores compiled wiki pages and source-state metadata.
This separation matters because each layer changes for a different reason. A project briefing changes when the project changes. A skill changes when the procedure changes. A rule changes when a boundary becomes non-negotiable. A wiki page changes when new source material has been synthesized.
The Dispatcher
Many local agents can read a user-level instruction file. Agent Harness uses that file as a dispatcher, not as a dumping ground for all knowledge.
The dispatcher should stay small. It tells the agent:
- where user profile and preference files live
- how to find project briefings
- where hard rules are stored
- when to read the LLM Wiki
- what must always be treated as source of truth
This prevents every session from loading unrelated information while still giving agents a reliable path into the right context.
Skills And Workflows
A skill is a reusable capability description. It should say when it applies, what to read first, which commands or tools are safe, and how to verify the result.
A workflow is broader. It describes a multi-step process across systems. For example, a content-aware LLM Wiki workflow may include:
- remote source-state refresh
- local pending queue inspection
- local raw export into ignored staging
- local synthesis by an agent
- wiki linting
- source hash acknowledgement
- commit and push
This is not just a script. It is an operating procedure with safety boundaries.
LLM Wiki As A Compiled Knowledge Layer
Agent Harness can connect to a long-term content system, but the raw content should not be confused with the compiled wiki.
A safer model is:
- canonical source: the database or content management system
- raw staging: ignored local exports for processing
- remote state: source keys, titles, timestamps, status, and hashes
- compiled wiki: reviewed pages written for agents
- acknowledgement state: source hashes that have already been reflected in the wiki
With this model, a remote workflow can detect source changes without storing full article bodies or calling an LLM API. The actual synthesis happens locally, where the user can review and commit the result.
This keeps the content platform independent from any specific AI provider, while still letting the harness know when source material has changed.
Remote Awareness, Local Synthesis
A robust synchronization loop looks like this:
- A GitHub Action runs weekly, manually, or from a webhook.
- It performs read-only source queries.
- It writes a hash-only
remote-state.json. - It writes a
sync-queue.jsonof new, changed, or deleted source items. - A local agent pulls the harness and checks the queue.
- The local agent exports only the needed raw content into ignored staging.
- The local agent updates wiki pages.
- The wiki is linted.
- Processed source hashes are acknowledged.
- The harness is committed and pushed.
This approach deliberately avoids running LLM synthesis in GitHub Actions. The remote repository only tracks machine-readable state. The local environment performs the higher-context work.
Safety Principles
Agent Harness should be designed with conservative defaults:
- Ignore everything unless it is explicitly allowlisted.
- Do not commit tokens, keys,
.envfiles, logs, caches, or local tool state. - Do not store raw private content in the remote harness repository.
- Do not store generated summaries in remote source-state files.
- Keep raw exports in ignored local staging.
- Back up production data before remote writes.
- Keep workflows small, inspectable, and reversible.
- Review public-facing documents before publishing.
Public documentation should describe architecture and patterns, not private infrastructure. It should avoid personal details, internal hostnames, account identifiers, secrets, private repository paths, and operational credentials.
Why This Makes Agents Better
The value of Agent Harness is not magic memory. It is disciplined context engineering.
When the agent starts with a stable map of project facts, user preferences, safety boundaries, and reusable workflows, it spends less time guessing. It can distinguish source of truth from generated artifacts. It can run the right checks. It can reuse prior procedures. It can improve the harness after completing a task.
Over time, the system becomes a feedback loop: work produces knowledge, knowledge becomes structured context, structured context improves the next agent session.
That is the practical meaning of making local agents smarter: not giving them unlimited memory, but giving them well-organized, versioned, auditable context they can trust.