The structural spine: navigate your wiki by shape, not keywords

The problem: structural blindness

A wiki with 1,000+ pages and no structural API forces every agent to rediscover the wiki's shape via full-text search. An agent asked "what does this wiki know about retrieval?" must BM25-search for "retrieval", hope the top hit is a hub page, then parse prose to find sub-articles. An agent asked "show me every hub page" has no endpoint to call — it must enumerate all pages and re-parse their frontmatter client-side.

This is expensive in tokens and brittle in practice. The structural spine solves it by exposing the taxonomy as queryable data: clusters, tags, types, canonical IDs, and sitemap — all available as structured API responses, not prose that needs to be parsed.

The full design is documented in Structural Spine Design in the live wiki.

First-class structural APIs

The /api/structure/* REST endpoints expose the structural index:

• GET /api/structure/clusters — all clusters with hub page, article count, and last-updated timestamp
• GET /api/structure/clusters/{name} — full cluster detail: hub summary, ordered article list with summaries, tag distribution
• GET /api/structure/tags?min_pages=N — tag dictionary with counts and representative pages
• GET /api/structure/pages?type=hub&cluster=X&tag=Y&updated_since=ISO — filtered, paginated page listing
• GET /api/structure/sitemap?shape=compact — compact machine-readable sitemap suitable as an agent prelude payload (approximately 200 bytes per page)

The same data is available via the /knowledge-mcp MCP server's structural spine tools (list_clusters, list_tags, list_pages_by_filter, get_page_by_id), so MCP-native and REST-native agents share one surface.

Canonical IDs: rename-stable page identity

Every page carries a canonical_id: a 26-character ULID assigned at first save by StructuralSpinePageFilter and stored in the page's YAML frontmatter and in the page_canonical_ids database table. The canonical ID is immutable — renaming the page does not change it.

This means external systems that reference pages by canonical ID instead of slug are rename-proof. The endpoint GET /api/pages/by-id/{canonical_id} always resolves to the current page, even through multiple renames. Slug history is tracked in page_slug_history for auditing.

Canonical ID assignment is enforced at save time: if a page lacks a canonical_id, StructuralSpinePageFilter automatically assigns one before the save completes. You cannot accidentally save a page without a stable identity.

Cluster hubs: the entry points agents use

Every page belongs to a cluster declared via cluster: in its frontmatter. Within each cluster, one page is the hub — the entry point that links to all cluster articles. The structural index tracks hub designation per cluster, so list_clusters returns not just cluster names but the hub page's canonical ID, slug, and title.

For agents navigating a large wiki, the pattern is: call list_clusters to get a map of the wiki's topic areas; pick the relevant cluster; call get_cluster to get the hub page and article list; then read specific articles via read_pages or get_page_by_id. This is three to four tool calls to fully orient an agent in any topic area, versus unbounded keyword search iterations.

Generated Main.md: structure that stays accurate

Wikantik's Main.md — the wiki's homepage — is generated from the structural index rather than hand-maintained. The build pipeline runs the generator during the package phase, so Main.md is kept in sync with the actual cluster taxonomy. A CI regression test (MainPageRegressionTest) fails the build if the checked-in Main.md diverges from what the generator would produce.

Curators can add editorial pins via docs/wikantik-pages/Main.pins.yaml — featured pages and highlighted sections — without hand-editing Main.md directly. The generator reads both the structural output and the pins, merging them into the final page.

Fail-closed behaviour

The structural index follows the same fail-closed principle as hybrid retrieval. If the index is being rebuilt after startup, structural endpoints return HTTP 503 with a Retry-After header rather than silently returning stale or empty data. MCP tools return a structured rebuilding error with an ETA hint. An agent that gets a 503 from a structural endpoint knows to retry — it does not get silently wrong data.

Frequently asked questions

What problem does the structural spine solve?

Without a structural spine, an agent asked "what does this wiki contain?" must run a full-text search and hope for a useful hub page. The structural spine exposes the wiki's taxonomy — clusters, tags, types, canonical IDs — as queryable data. An agent can enumerate clusters, filter pages by type or tag, and resolve canonical IDs without guessing at keywords.

What is a canonical ID and why is it immutable?

A canonical_id is a 26-character ULID assigned to each page at first save by StructuralSpinePageFilter. It is immutable — renaming the page does not change it. External systems that reference a page by canonical ID instead of slug are rename-proof; GET /api/pages/by-id/{canonical_id} always resolves to the current page.

How is Main.md kept in sync with the actual wiki structure?

Main.md is generated from the structural index rather than hand-maintained. The build pipeline runs the generator during the package phase, so Main.md is kept in sync with the actual cluster taxonomy. A CI regression test fails the build if the checked-in Main.md diverges. Curators add editorial pins via Main.pins.yaml without hand-editing Main.md directly.