Control Plane — Go server and MCP tools¶

Rationale¶

Go: static binary, performant on I/O across many files, containerizable. MCP: an abstraction layer that keeps the system agnostic to the agent (Claude Code, Codex, Kiro, OpenCode all speak the same protocol).

A (nearly) stateless server¶

For each mounted KB, the server keeps only a rebuildable derived index and a parsing cache. Everything else lives in the files. Multiple KBs per instance: each with its own index, lock, and git repo.

Read/write boundary¶

Concepts are read and written through MCP tools. Writes validate parseable frontmatter, required fields, path/layout constraints and strict-map type palettes.
if_match protects updates from stale overwrites.
With git auto-commit enabled, each successful logical write produces one commit. A write does not implicitly append to log.md; use log_append when a chronological record is wanted.
Read-only tools carry Tool.ReadOnly=true; HTTP scopes enforce the read/write boundary.

Commit per logical operation (Step 1 — local commit): every write tool (concept_write, concept_new, concept_patch, map_create, map_delete, concept_expand, asset_write, asset_delete, log_append, snapshot, supersede, concept_move, concept_delete, conflict_resolve, skill_install) is wrapped by gitWrap, which acquires the per-KB mutex, runs the tool, and on success (no application error) calls CommitOp. A failed commit does not turn a successful operation into an error: it is logged to stderr. AutoCommit=false (the struct's zero value) leaves everything unchanged and keeps compatibility with existing tests.

Read freshness across instances (D93): when git.sync is enabled and a KB has an origin, every tool marked [R] piggybacks a fetch + pull-rebase before handling the read, at most once per git.in_window per KB. A pull that moves HEAD reconciles the derived indexes before the read, so a remote concept becomes both readable and searchable. Read-side sync is best-effort: a fetch error is logged and serves the local replica; a rebase conflict is registered and its concepts are marked degraded, then the read still serves the local tree. There is no background poller.

MCP API¶

This list is the source of truth for the active tools (do not duplicate counts elsewhere).

Tools marked [R] have Tool.ReadOnly=true (internal/mcpserver): they never mutate KB content and remain callable with a kb:<name>:r scope. All others require rw. Enforcement and scope format → transport-auth.md §Per-KB authorization.

Beyond that read/write split, every tool is also classified by the resource it addresses — exact concept, collection, source/destination, or whole KB — which is what a fine-grained role is evaluated against (D118). Collection tools filter their results per element before applying any limit; exact-resource tools collapse "forbidden" and "missing" into the same generic not found. A tool missing from that classification is denied outright, so adding a tool without choosing its resource semantics fails closed rather than granting access. The table and the guarantees are in transport-auth.md §Roles and fine-grained permissions. The same [R] tools expose annotations: {"readOnlyHint": true} in tools/list per the MCP spec (D76): a client can use this to auto-approve reads without a manual allowlist, without having to derive the list by hand.

Tools marked [A] (advanced, advancedToolNames in internal/mcpserver/visibility.go) are hidden from tools/list in the default agent tool profile (D65): governance/maintenance and provisioning plumbing that would bloat the LLM agent's context without being useful in a normal session. They all remain callable via tools/call by name (CLI client, hooks, operator) in both profiles. Profile: tools.profile in YAML / CARTOGRAPHER_TOOLS_PROFILE / --tools-profile, values agent (default) | full → deployment.md.

Tool	Purpose
`atlas_overview()` [R]	Root index + maps/journals (concept count, and expanded concept count if any).
`map_list()` [R]	Lists maps and journals with metadata (`kind`, `ontology_mode`, `concept_types`, expanded concept count).
`index_get(path)` [R]	Reads a folder's `index.md` (progressive disclosure).
`concept_read(id, [section], [outline], [full])` [R]	Reads a concept or a single section (bounded). Returns the content-hash for `if_match`. A `section` not found → error with the list of available headings (capped at 50), no guessing. `outline=true` returns only the structure (`{level, title, bytes}` per heading) with no content. Size guard: a body over 60 KB without `section` or `outline` → returned as `outline` plus a `note` (no content), unless `full=true` forces the full content (D78).
`log_tail(path, [n])` [R]	Latest N entries relevant to `path`. Empty `path` = the verbatim root log. A non-empty `path` has no `log.md` of its own (see `log_append`): it filters root entries prefixed `[<path>]`, preceded by any entries of a pre-existing `<path>/log.md`. No entries → a JSON note `{"entries": 0, "note": "..."}`, never a silent empty string (D78).
`changes_since([since], [limit])` [R]	Git-history digest since an RFC3339 timestamp or `<N>d`/`<N>h` duration (default 7d). Returns each changed concept's newest status (`added`, `modified`, `deleted`, `moved`), latest timestamp, authors and recent operations; map/log/artifact changes are counted as `other_changes`. Results default to 100 concepts (maximum 500) and report `truncated`; no commits returns a JSON note rather than an empty string (D94).
`graph_neighbors(id, [depth], [direction])` [R]	Graph neighbors (used for lint scoping). `direction` is `out` (default, links from the page), `in` (backlinks to it), or `both`; traversal returns minimum distances and can start from a missing target for backlink inspection. The graph sees both markdown links `[text](rel.md)` (relative to the file) and wiki-links `[[id]]`/`[[id#section]]` (root-relative, D72 WP0). Ask `direction: "in"` before editing a runbook to see which pages cite it.
`concept_list([scope], [limit], [where], [timestamp_before], [timestamp_after])` [R]	Exhaustive inventory: `{id, title, type}` for every concept under the `scope` prefix (empty = the whole KB, including `services/`), ordered by id. `where` is an ANDed list of case-sensitive `key=value` / `key!=value` frontmatter predicates (a list matches any element; a missing key matches `!=` only); timestamp bounds are strict RFC3339 or `YYYY-MM-DD` comparisons. Filters run before `limit`; filtered responses report `examined`, and timestamp queries also report skipped timestamps. Ask `where: ["tags=istio"]` to list everything tagged `istio`. `limit` defaults to 500, with `truncated`/`total` if exceeded. An exhaustive alternative to `index_get`'s progressive disclosure (D72 WP3).

Search¶

Tool	Purpose
`search(query, [scope], [mode])` [R]	Keyword, semantic, or hybrid (keyword + vector) search. `mode` is `keyword` (default), `semantic`, or `hybrid`; semantic/hybrid require Ollama to be configured. `use_semantic=true` remains a deprecated alias for `mode=hybrid`. Keyword matching first requires all query terms, then retries with any term if that returns no hits. Every hit includes `title` (from the frontmatter) and `snippet` (an excerpt of ~200 chars around the match; FTS5 uses its native `snippet()`, otherwise it's extracted in-memory) — avoids a `concept_read` just to judge a hit's relevance (D70).
`index_rebuild()` [R] [A]	Rebuilds the keyword index (in-memory and FTS5 if present) and the embeddings if Ollama is active (with a content-hash cache). Read-only: it mutates only the derived/gitignored index, never KB content (D45).
`reindex()`	Reconciles the derived FTS5 and in-memory search indexes with concepts changed outside MCP. Use after imports or manual edits; returns `indexed`, `updated`, and `removed`. It requires write scope because it writes the server-owned SQLite index (D90).

Writing and ingest¶

Tool	Purpose
`map_create(name, title, [kind], [concept_types], [ontology_mode], [required_fields], [required_fields_by_type], [require_index_entry])`	Creates a map (`kind: map`, default) or a journal (`kind: journal`): a directory with `_map.md`, `index.md`, `log.md`. The optional contract fields are serialized deterministically in its descriptor.
`map_delete(map)`	Deletes a map/journal directory, but only if it holds nothing beyond the `map_create` scaffold (`_map.md`, `index.md`, `log.md`); if any concept remains, errors listing them — move them out with `concept_move` first, then retry (D88).
`concept_expand(id)`	Promotes a concept to an expanded concept: `map/name.md` → `map/name/index.md`, same ConceptID (no backlink rewrite), from which it can grow with `map/name/child` satellites. Requires a 2-segment id; errors `not_found` / `already_expanded`. No inverse operation (D77).
`asset_read(concept_id, path, [encoding])` [R]	Reads a non-Markdown asset inside an expanded concept. Returns content (`text` or base64; invalid UTF-8 is always base64), raw-byte `sha256`, size and executable mode.
`asset_list(concept_id)` [R]	Lists an expanded concept's non-Markdown regular assets with path, size, raw-byte `sha256`, and executable mode.
`asset_write(concept_id, path, content, [encoding], [executable], [if_match])`	Creates or updates an expanded concept asset. 1 MiB decoded limit; `.md`, hidden, escaping and symlink paths are rejected. `if_match` is forbidden on create and mandatory on overwrite; omitted `executable` defaults false on create and preserves mode on overwrite.
`asset_delete(concept_id, path, if_match)` [A]	Deletes an asset with mandatory raw-byte `if_match`, pruning empty asset-only directories but never the owner directory.
`concept_write(id, frontmatter, body, [mode], if_match)`	Creates/updates with validation. `if_match` = expected content-hash; fails with `stale_write` if changed. Automatically updates the in-memory keyword index and, if present, the persisted FTS5 index (no `index_rebuild` needed; embeddings remain `index_rebuild`'s job).
`concept_new(template, id, [vars])`	Creates a new concept from `templates/<slug>.md`, with literal single-pass `{{identifier}}` substitution in frontmatter values and body. Refuses existing IDs and missing/extra variables; updates indexes and makes one commit. It creates only the concept (no curated index entry) and does not pre-check a strict-map ontology because templates can serve multiple maps.
`concept_patch(id, old_string, new_string, [replace_all], if_match, [frontmatter])`	String-replace patch on the body only (Edit-like semantics), without rewriting the entire concept. `if_match` is mandatory: fails with `stale_write` if changed. Fails with `old_string_not_found` or `old_string_ambiguous` (use `replace_all` for multiple matches). `frontmatter`, if present, is shallow-merged onto the existing frontmatter; a key set to `null` is removed rather than set to a literal null (fails if the key is required, e.g. `type`, D88). Same write path (indices, commit) as `concept_write` (D70). As an alternative to the `old_string`/`new_string`/`replace_all` triple, it accepts an `edits: [{old_string, new_string, replace_all?}]` field to apply several patches in a single call (one commit): the two forms are mutually exclusive; edits are applied in order, atomically (edit i+1 sees the result of edit i) — if one fails, nothing is written and the error reports the index of the failed edit (D76).
`log_append(entry, [path])`	Appends an entry to the root log (never a per-directory log). With `path`, the entry is prefixed `[<path>]` and still written to root; `log_tail(path)` retrieves it by filtering on that prefix (D78).
`snapshot([message])`	Records an entry in `log.md`. With git auto-commit enabled, it also creates a git commit of the entire KB.
`supersede(source_id, target_id, [reason])`	Marks a concept as superseded by another.
`concept_move(source_id, target_id \\| moves[], [rewrite_links])`	Moves one or more concepts (batch `moves: [{source_id, target_id}]`, single form kept for backward compatibility; the two forms cannot be mixed). Validates the entire batch before applying any move (application-level atomicity), one git commit per call. With `rewrite_links` (default `true`) it rewrites backlinks across the whole KB — wiki-links `[[old]]`/`[[old#section]]` and relative markdown links — in a single pass using the old→new map, and updates the indices (in-memory + FTS5) for both the moved and rewritten concepts; the result lists the applied moves and rewritten concepts. With `rewrite_links=false` backlinks are left intact (warning in the result, use `lint`). D72 WP1/WP2.
`concept_delete(id, [if_match], [force])`	Permanently removes a concept from the KB (git commit). An expanded owner with assets refuses deletion until `force: true`; force deletes assets and the owner index but preserves satellite Markdown concepts. Incoming backlinks are not updated — use `lint` to find them.

Governance¶

Tool	Purpose
`validate(scope)` [R] [A]	OKF compliance (frontmatter, `type`, reserved files).
`lint([scope], [scope_neighbors])` [R] [A]	Runs deterministic broken-link, stale-claim, orphan, contract and structural checks. `scope_neighbors=true` adds one-hop graph neighbors. There is no model-backed/deep mode.
`commit_gate()` [A]	Blocks when open `Contradiction`s are involved in the diff.
`gate_check()` [R] [A]	Combines validate + lint + commit_gate in a single tool (lightweight local gate).
`conflict_resolve(contradiction_id, resolution, [reason])` [A]	Closes an open `Contradiction`.
`contradiction_report([scope], [status])` [A]	Lists contradictions, filterable by scope and status.
`kb_status()` [R] [A]	Aggregate metrics: total concepts, per type, stale ones, open contradictions.
`conflicts_list()` [R]	Lists open git rebase conflicts (read-only). For each entry: `concept_id`, local/remote SHAs, `branch`, files involved, `detected_at`, resolution guidance. See also the `kb-conflict-resolve` skill.
`git_conflict_resolve(concept_id, strategy, [body])`	Resolves a registered conflict (Step 4). `strategy`: `ours` (local version), `theirs` (remote version), `edit` (full content in `body`). Records the per-concept decision; once every open conflict is resolved, it performs a single merge+commit+push and clears the `degraded` markers. See `concurrency.md` §Step 4.

Skills and Services¶

Tool	Purpose
`skill_list()` [R] [A]	Lists the installed skills (`skills/`) and the ones bundled in the binary. Field `source`: `[installed]` \| `[bundled]`.
`skill_install(name, force)` [A]	Copies a bundled skill into `kb.Root/skills/<name>/`. Errors if already present; `force=true` overwrites.
`service_get(service_id, resolve_secrets=false)` [R] [A]	Reads a Service. With `resolve_secrets: true`, resolves declared `secret_refs` (or legacy whole `secrets_source`) and requires rw scope.
`secret_resolve(concept_id, names?)` [RW] [A]	Resolves SOPS references declared by any concept; `names` narrows the declared values.
`secret_set(path, key, value)` [RW] [A]	Sets a JSON Pointer in an existing encrypted SOPS file; plaintext is never committed or returned.
`service_list()` [R] [A]	Lists all concepts of type Service.

Client ↔ provisioning synchronization¶

Tool	Purpose
`sync_check([applied_revision])` [R] [A]	Read-only. Returns the current manifest's revision and artifact list, including cryptographic `signed` verification output and separate `built_in` trust origin.
`sync_apply(base_dir, [dry_run], [auto_trust])` [A]	Materializes verified or built-in artifacts into `base_dir`; `auto_trust` is explicit unsigned authorization and never sets `signed`. A local signer is verified before applying.
`sync_pull()` [R] [A]	Read-only, no parameters. Returns base64 file contents plus optional detached Ed25519 signature (`algorithm`, `key_id`, envelope version, value). Remote clients recompute hashes and verify pinned keys before materialization. KB-provided MCP descriptors appear only when the server-side allow-list permits them.
`sync_status()` [R] [A]	Read-only. Returns local Git replication state (`disabled`, `no_remote`, `clean`, `pending`, or `failed`), the last error/attempt, HEAD, best-effort unpushed count, identity warning, and push mode.
`pr_status()` [R] [A]	Read-only, server Git profile only (D117). Returns the profile, base/working branches, current PR identity and last forge error; reconciles a `merge_uncertain` state first. Never returns the forge token.
`pr_finalize(head_sha)` [A]	Server Git profile only (D117). Squash-merges the open PR after checking that `head_sha` matches the current remote head, reviews/checks are satisfied, and a fresh rebase plus validation succeed. Force-with-lease touches only the working branch; the base is never pushed.

KB-root artifacts¶

Tool	Purpose
`artifact_read(path, [encoding])` [R]	Reads a KB-root artifact file (`skills/<slug>/`, `agents/<slug>.md`, `hooks/`, `mcp/<slug>.json`, `instructions.md`, `templates/<slug>.md`) and returns its content, `encoding` (`text` or `base64`) and raw-byte `sha256` (the `if_match` for `artifact_write`). Non-UTF-8 bytes are always base64.
`artifact_list()` [R] [A]	Lists provisioning artifacts and KB-only templates by kind, with files, raw-byte sha256 and executable metadata. Templates are scanned directly because they are absent from `provisioning.BuildManifest`.
`template_list()` [R]	Agent-visible template discovery: returns each template's slug, literal `type`, title and sorted variables, never its body.
`artifact_write(path, content, [encoding], [executable], [if_match])`	Creates/updates a KB-root artifact file. `encoding` is `text` (default) or `base64`; the 256 KiB limit applies after decoding. `executable` is tri-state: omitted preserves an existing mode and defaults false on creation. Structured principal files remain UTF-8 and non-executable. Templates require parseable frontmatter with a literal non-empty `type` and only `{{identifier}}` variables in scalar/list values or body. On an existing file, `if_match` (raw-byte sha256) is mandatory (`already_exists` if missing, `stale_write` if wrong); per-kind validation applies before the write. Only registered if the KB has `allow_artifact_write: true`.
`artifact_delete(path, if_match)` [A]	Removes a KB-root artifact file (and an eligible empty directory). Same per-KB flag as `artifact_write`.

artifact_write/artifact_delete go through gitWrap (lock, commit, sync) and notify skills/list_changed when the path is under skills/. The per-KB flag → deployment.md.

See docs/sync.md for the full model (Manifest, Lock, Diff, layered triggers).

Multi-KB: which KB a tool call reaches is decided per-connection, not per-call — ?kb=<name> (query param) or /mcp/<name> (path) select one KB's isolated Server for the whole session; no tool takes a kb argument. Every KB exposes the same tool names by default, which flat-namespace MCP clients (e.g. Kiro) cannot disambiguate across servers — see tool_prefix in deployment.md §MCP tool-name prefix.

Semantic search¶

Semantic search is available when the server is started with --ollama <url> (or CARTOGRAPHER_OLLAMA=<url>). In that case:

The search tool accepts mode=semantic or mode=hybrid to use vector similarity; use_semantic=true remains a deprecated alias for mode=hybrid.
The index_rebuild tool rebuilds the keyword index and per-concept embeddings.
The model is configurable via CARTOGRAPHER_OLLAMA_MODEL (default: nomic-embed-text).
If Ollama is unreachable or embedding fails, keyword search is always available as a fallback.

Search index¶

Rebuildable index (vault = truth, index = disposable), with two persistence levels:

In-memory (default/Core): a pure-Go keyword inverted index (internal/search) + an in-memory vector store (internal/embed). Rebuilt on every startup by walking the concepts.
Persisted SQLite (internal/sqlindex, D32): when the KB has an openable .cartographer/index.db, keyword search uses FTS5 with a trigram tokenizer (supports substrings, not just whole words). Multi-term matching tries all terms first, then any term only if the all-terms search is empty; terms shorter than three characters are omitted from the FTS match. Embeddings are persisted in SQLite with a content-hash cache — index_rebuild recomputes Ollama embeddings only for concepts whose hash changed. At boot, after a git pull that moves HEAD, and on reindex, content hashes reconcile external additions, changes, and removals without a restart. On this path, the search response's mode field is keyword_fts5/hybrid_fts5. Best-effort: if the DB can't be opened or FTS5 is unavailable, it degrades to the in-memory path.
Semantic: embedding vectors (e.g. Ollama), cosine similarity outside SQL. Independent commit: if the embedder is down, keyword search still moves forward. Embedder-identity guard: full re-embedding if the model changes (content-hash invalidates the cache).

At small scale index.md and keyword search can be enough. Semantic search is opt-in and active only when an Ollama endpoint is configured.

Validation and invariants¶

validate checks parseable frontmatter, a non-empty type, reserved files, layout/depth rules and the allowed type palette of strict Maps.
lint reports deterministic findings. It does not generate typed graph edges or Contradiction concepts.
Contract findings are missing_required_field (error), index_incomplete (warning), and contract_malformed (info). A lint error makes gate_check fail; commit_gate remains contradiction-only.
commit_gate inspects existing type: Contradiction concepts with resolution_status: open and blocks a supplied set of changed concept IDs when they are involved.
Broken links are tolerated on write and surfaced by lint.

Provenance, history and audit¶

Provenance and citation sections are KB conventions; the server does not require them on every concept. log.md is updated only through explicit log tools. Git commits are the durable change history when auto-commit is enabled.

internal/audit implements a JSONL hash-chain with optional Ed25519 signatures. When audit.log is configured, every tools/call appends an attempt event before dispatch and a completion event after it (D119), so the log is a complete operational audit trail: semantics, failure modes and the cartographer audit verify|export commands are in transport-auth.md §Operational audit.

The content hash is computed on normalized content and per section to avoid spurious stale_write failures.