Skip to content

Workbench interface: technical spec

  • Status: sliced, 2026-10-12. The slices are in epic #113; §11 and §12 below were amended by that grill. Everything else is unchanged.
  • Why: ADR 0001
  • Decisions: interface-redesign.md, D1–D18. This spec cites them as (D#).
  • Prototype: scripts/dashboard_static/prototype-shell.html, served at /prototype. It's throwaway: delete it in slice 9.

How to read this spec. Every requirement is one of three kinds:

  • Decided — cites a decision (D#) or an earlier plan decision (P#, A#, T# in dashboard-ux-plan.md). Changing it means revisiting that decision.
  • Default — a choice made while writing this spec. It hasn't been separately confirmed; change it freely in a slice.
  • Verify — an unknown. Check it against the real system before building on it. The full list is in §13.

This repo is public, so nothing here names real projects, spaces, clients or people. Examples use made-up names.


1. Constraints that apply everywhere

  1. Models run only on explicit action. Nothing is generated in the background. Before any model call is sent, it shows its model, its route and the data classification it is working on. (D2)
  2. Routing policy, enforced on the server before any run starts; see §8. (D17)
  3. Security. The server stays bound to 127.0.0.1. Host is checked on every request, and every POST needs this origin plus the per-launch token (scripts/dashboard.py, request_rejection). Every endpoint that can start an agent has tests for all three checks. (D6)
  4. Works for anyone who clones it.
  5. Every surface renders with an empty data root and with any service missing (AnyType, Ollama, OmniRoute, the tracer), and has a designed empty state.
  6. No personal data (space names, project lists, topics) in front-end code or in tracked config. (D7)
  7. One status vocabulary, used by every surface. §4.3 defines it.
  8. Private state stays private.
  9. Everything new goes under the data root, never in the repo. (P1a, D14)
  10. Each machine is its own island: nothing syncs between machines. (D16, D17)

2. Architecture

browser (installed web app, D6)
  └─ /next/...  Preact + htm shell (D11)
       ├─ GET  existing JSON endpoints (unchanged)
       ├─ GET  new read endpoints (§6)
       ├─ POST new action endpoints (§6)   ← Host + Origin + token
       └─ GET  /api/next/stream/<run>      ← server-sent events

scripts/dashboard.py  (standard-library ThreadingHTTPServer, unchanged model)
  ├─ existing modules (projects, tasks, automations, clones, search, health …)
  ├─ runner.py        starts and supervises claude -p / prompt-mode calls (§7)
  ├─ policy.py        routing policy: load, narrow, decide (§8)
  ├─ run_index.py     SQLite: runs, experiments, conversations, budget, UI state (§9)
  ├─ lab.py           experiments, variants, checks, promote (§5.4)
  ├─ standin_mcp.py   stand-in MCP server for Lab runs (§7.3)
  └─ approvals        permission prompts from claude -p surfaced to the UI (§7.2)

claude -p ──OTLP──▶ [tracing].endpoint (Phoenix, locally, to start; D15)
  • Decided: keep the standard-library server. It gains a small set of modules; there is no web framework. (D11, D14)
  • Default: new modules go under scripts/, next to the ones they use, each with a test_*.py wired into the Makefile the same way as today (test_tests_are_wired.py enforces this).

3. Front end

3.1 Stack (D11)

  • Preact + htm, pinned, committed as single-file ES modules under scripts/dashboard_static/next/vendor/. There is no build step. Each file's license header is kept, and its version and SHA-256 go in vendor/VENDORED.md.
  • CodeMirror 6 exception. The Lab editor uses CodeMirror 6, bundled once into vendor/codemirror.bundle.js. The one-off command that built it is recorded in vendor/VENDORED.md with the exact package versions. Updating it is a deliberate, reviewed change.
  • Types: JSDoc annotations. tsc --noEmit --allowJs --checkJs is optional and not a gate.
  • Tests (today's front-end "tests" only match strings in dashboard.js):
  • Pure logic (status mapping, policy display, URL parsing, the store) has unit tests on Node's built-in node:test, run only if Node is present. Default: skipped with a notice when it isn't.
  • Browser checks use Playwright for Python, a uv dev dependency, so no Node toolchain is needed. They run against a server with a fixture data root. Default: every surface gets a smoke test (renders, empty state, one action) and a no-console-errors check.

3.2 Layout on disk (default)

scripts/dashboard_static/next/
  index.html            shell document; carries {{DASHBOARD_TOKEN}}
  app.js                entry: router, store, shell
  store.js              app state + persisted UI state (§9)
  api.js                fetch wrapper: token header, error shape, SSE helper
  status.js             the status vocabulary (§4.3)
  shell/                rail, selector, sidebar, tabs, status bar, palette, ask panel
  surfaces/             today, projects, lab, library, agents, settings
  vendor/               preact, htm, codemirror bundle, VENDORED.md
  app.css               tokens + components (dark first; see §3.4)

3.3 Routing and URLs

  • Decided: the selected project is in the URL. (D9)
  • Default: paths look like /next/<surface>[/<item>]?project=<name>.
  • Examples: /next/library/skill/project/pkm-capture?project=acme-portal, /next/agents/traces.
  • The server serves the shell for any /next/ path. The client router maps the path to a surface and item.
  • An unknown path or project falls back to Today with a notice.
  • "All projects" is the absence of ?project=. (D9)

3.4 Visual language (default)

Tokens are taken from the prototype: the dark Workbench palette, 13px UI text, a monospace face for code and ids, and status colours reserved for status. A light theme is a later slice. It must use the same tokens, redefined, and must not flip colours automatically.

3.5 List views (decided 2026-10-06)

Every list of items (repos, runs, tasks, skills and so on) has a Sort control. The default is alphabetical, ascending, by the item's name, unless the list has a natural order, which is then its default (run history, newest first, is the obvious case). The chosen sort applies to the whole list, not to a page of it, and the user can always switch to name order.


4. Shell

4.1 Frame (D1, D5, D8, D9, D12)

Part Behaviour
Rail (left) Today, Projects, Lab, Library, Agents; Settings at the bottom. Today shows a badge when anything is in warn or fail.
Project selector (top of the sidebar, like the Google Cloud console) Entries: All projects, each directory under projects/, New project…, Manage projects…. Changing it rewrites ?project= and re-filters every surface. Browsing a project hub does not change it. (D9)
Sidebar An explorer that changes with the surface (§5). Has a filter box where lists are long.
Tabs One tab per surface; the label names the current item, e.g. "Library · pkm-capture". Tabs can be closed, and they survive a reload (§9, server-side). (D8)
Status bar Current project, key services with the status vocabulary, today's spend (from the run index), live/loading. Clicking an item goes to its surface.
⌘K palette Go to any surface or item, switch project, search the Library, capture, Ask, run a Lab experiment.
Ask panel (⌘J) §4.2.

Surfaces the selector doesn't scope (Settings, Automations, most of the Library) show a "not project-scoped" marker. (D9)

4.2 Ask panel (D12)

  • A right-side panel on every surface. It stays open while you navigate and can widen into a tab.
  • Header, editable before sending: model, route and classification.
  • The route list is filtered by the routing policy (§8).
  • The classification defaults to the current project's (§8.3).
  • Context chips: whatever you are looking at is attached as removable chips (the project, a Library passage, a Lab trace, a skill). Nothing is attached without a chip.
  • Thread:
  • The answer streams in over server-sent events.
  • Tool calls are collapsed by default.
  • Write-gate approvals appear inline as Approve / Deny (§7.2).
  • Errors are shown in the thread, never swallowed.
  • Follow-ups continue the same session. Open in Claude Code copies claude --resume <session-id> for a terminal.
  • Every conversation writes a run row (§9), so it appears in Agents → Traces.
  • Capture (text or voice) is reached from ⌘K and from the panel, and always asks for a classification before filing. (Existing capture rules: A1–A3, T1.)

4.3 Status vocabulary (default names, decided need)

State Meaning Glyph
ok working, recently confirmed ● green
warn needs attention, still running ▲ amber
fail broken ■ red
off deliberately not running (paused, not installed, disabled) ○ grey
notrun never checked or never run ◌ grey
info worth knowing, not a problem (clones behind upstream) · grey

Mapping rules:

  • An automation that is defined but not installed is off, never ok.
  • /api/health values map as: ok→ok, fail/error→fail, unknown→notrun, anything else→warn.
  • Every state has a glyph and a word, so colour is never the only signal.

5. Surfaces

The "data" column says where each surface's data comes from. E = an existing endpoint, unchanged. N = a new endpoint (§6).

5.1 Today (D1; round-2 verdict)

A briefing, read from top to bottom:

  1. Summary line: date, greeting, and one sentence of counts, e.g. "2 things need a look, 1 task is due today, and 4 projects have uncommitted work". Each count scrolls to its section.
  2. Needs attention. One list, most severe first.
  3. It holds failing services, automations in warn/fail, automations defined but off, diverged clones, and clones behind upstream (info).
  4. Every row carries its own specific actions (install all, update N, show local edits, reset to upstream…).
  5. Commitments: open tasks with due state, a done checkbox, due…, project…, and an inline "add a commitment".
  6. Where you left off: recently changed projects (filtered by the selector), with open / terminal / diff.
  7. Inbox: untriaged captures.

Glance column: Systems, Spend today plus the budget state, the current Lab experiment, and recent agent runs.

Sidebar: Views (Today, This week — later), Inbox, Pinned, Open tabs.

Data: E /api/health, /api/tasks, /api/automations, /api/clones, /api/projects. N /api/next/runs/summary, /api/next/budget, /api/next/ui.

Budget (confirmed): at ≥80% of the monthly cap, Needs attention gains a warn row. At 100%, Lab batch runs are held (§5.4); Ask still works, and says the budget is spent.

5.2 Projects (D5, D9; P1–P5)

  • Index: a table of every directory under projects/ with branch, working-tree state, last commit, domain, per-project connections and remote. It has a New project action, which keeps P4's thin create flow: name, domain, optional git init, no remote.
  • Hub for one project:
  • A header with Open in Cursor, Terminal, New experiment here and Ask….
  • Pills for branch, working tree, last commit and domain (or "not opted in · opt in…"), plus state-file mismatches.
  • Cards:
Card Content
Repository branch, tree, last commit, remote; diff; "rename / move…" shows make layout-plan, run with Claude Code and Cursor closed (P4)
Agent connections this project's clients × toggleable servers; globals marked "global"; toggles use P2's mechanism
PKM object the AnyType project object found by exact name in the domain's space (P5); create on demand
Commitments tasks linked through linked_projects (T1, P5); + task
Lab experiments experiments scoped to this project; + experiment
Relevant patterns from both pattern tiers (A5); an honest empty state while the corpus is empty
Agent runs here from the run index, filtered to this project

Data: E /api/projects, /api/projects/objects, /api/projects/create, /api/projects/toggle, /api/projects/object, /api/tasks. N /api/next/runs?project=, /api/next/lab/experiments?project=.

5.3 Library (D8)

One place for reading material and tools.

  • Find: search across the selected corpora (wiki, patterns, research, books, code library), as a list next to a reader.
  • The reader shows the result in the existing read-only viewer (/view), in a sandboxed iframe with no allow-same-origin: the iframe's sandbox attribute is allow-scripts and never allow-same-origin, and every /view response also sends Content-Security-Policy: sandbox allow-scripts (#92), so a viewed page has an opaque origin and can't read the per-launch token or call the dashboard's API. allow-scripts stays because the viewer's own Copy path and View source buttons are script (a bare attribute would block them, and the header already grants the same; settled building L1, #122).
  • Each result has capture, ask… (attaches the passage as a chip) and cite.
  • Read: Research, Books, Agent wiki and Patterns, each a search scoped to that corpus. Patterns shows its tier on every hit (#54, A5).
  • Staged review (thin, v1; #112): every entry the research scan held back (and, later, agent-suggested URLs) on one page, selectable with checkboxes, to promote or delete. Staged text is hostile by definition: it is shown as escaped text only, in the isolated /view origin, and no agent tool can list, promote or delete it. v2 may expand it.
  • Tools:
  • Skills: every skill in project and user scope, rendered, with Open in Lab, Edit and Duplicate.
  • MCP servers: from tool-chest.toml, with env keys only and smoke-test.
  • Code library: freshness, drift, local edits and fast-forward update, and Ingest (below).

Code library: Ingest (settled from the #20 pilot of the old dashboard, 2026-10-06). Ingest is how a cloned repo under the source root becomes searchable by agents. It is a two-stage flow, with the repo as the unit of work:

  • Repos stage: one table of every repo under the source root, sorted by name ascending unless changed (§3.5).
  • Each row shows its category, the suggested kind with its reasons, the confirmed kind (blank until confirmed), and a count of accepted entries.
  • Kinds are confirmed in bulk or per row, in the table. Nothing is hidden behind a separate panel, and the "queue" is the table itself.
  • Suggestions are computed for every row up front. The suggestion needs the monorepo-aware root discovery (below), or a monorepo shows a wrong suggestion.
  • Proposals stage: the confirmed repo's proposals.
  • Accept is one click and the default action. It accepts every proposal for the repo, with the repo's own category and no new categories. Collections group proposals by shape, with accept-all per group.
  • Confirming a kind never accepts anything. Confirming is a classification. Accepting is the step that makes content searchable, so the two stay separate.
  • Before the click, the button shows what becomes agent-searchable, such as "index 324 docs, 234 examples, 2,575 source files". That is the real trust decision: accepting any entry for a repo makes the whole repo's docs, examples and source searchable through the MCP, so per-unit approval is a weaker gate than it looks.
  • After accepting, individual entries can be pruned and categories edited. Nothing in an accepted recipe is ever executed: the scan is read-only and dry-run validates paths only.
  • Every action says when it has happened, whether a scan is pending or an entry was accepted.
  • Kinds: one kind per repo (tool, collection, library).
  • A repo with both a library and a CLI is a library. Its CLI is not offered through tool search, and its docs stay searchable.
  • Suggestion rules: the fallback is library, never tool, because tool with no entry point yields no proposals. Only the repo root decides tool. The reasons list manifests found to a bounded depth, such as "8 packages under libs/".
  • A repo that is both is a possible later extension (a repo with more than one kind), not part of v1.
  • API symbols come from Python packages only. A Rust repo shows none, which is a documented limitation and not a defect.

Data: E /api/search, /view, /api/library, /api/skill, /api/mcp, /api/clones (+ check/update), /api/code-library, /api/capabilities (corpus counts). N a batch-accept endpoint for Ingest, named when slice 4 is filed.

Absorbs: #54 (patterns tiers in search), #56 (prior-art view, as "ask… about this capture's notes"), #53 (attachments reachable in the viewer).

5.4 Lab (D4, D10, D13)

  • Experiment: artifact × variants × inputs × models, with named checks.
  • Artifact kinds: a skill (v1), then patterns and rules. Harnesses are a later candidate (#67).
  • Scope follows the selector. Setup shows the scope prominently and offers "make general". (D10)
  • Classification: every experiment carries one. It limits which models can be picked (§8).
  • Layout:
  • Editor pane: CodeMirror, one tab per variant, with a diff against the baseline.
  • Run grid: variant rows × model columns. Each cell shows checks passed, cost and time.
  • Trace pane: the selected run's timeline.
  • Actions: + variant, setup… (inputs, checks, models), Run N, Promote.
  • Modes:
  • Prompt: a direct model call with the artifact as text, across any allowed provider including Ollama. It tests whether a model follows the text.
  • Agent: isolated claude -p (§7.1). It tests triggering, file loading and tool use.
  • Checks: named assertions over a run, written as JSON in the experiment.
  • Default v1 kinds: triggered (the skill loaded); tool_called (name, with an optional argument match against stand-in call records); no_tool_called; asked_user (a scripted user turn was consumed); text_matches (a regex over the final text).
  • Each check records pass, fail or notrun, with evidence.
  • Promote: copies a variant into shared/skills/<name>/ on a new git branch. The change always goes through a commit you review. On a machine marked work, promote needs an explicit confirmation, until the employment-agreement question is checked (D17).
  • Budget: once the monthly cap is reached, Run N is held with a reason. A single run can still go ahead after a confirmation.

Data: N /api/next/lab/* (§6).

5.5 Agents (D3, D8, D15)

  • Traces: the run table from the run index, with filter chips (All, Ask, Lab, Automations, Problems) and a project filter from the selector.
  • The selected run shows model, route, classification, project and whether prompt text was recorded, plus a span timeline.
  • The timeline comes from the tracer when one is configured. Otherwise it is built from the run's own event log (§9).
  • Open trace ↗ uses [tracing].ui_url_template. Export JSONL exports the filtered rows.
  • Costs (the budget is confirmed):
  • Tiles: today, 7 days, 14 days, Lab share.
  • A daily spend chart: bars, one colour, a tooltip on every bar, no second axis.
  • Breakdowns by model, surface and route, and a monthly Budget setting.
  • v1 covers only dashboard-launched runs. (D15)
  • Automations: the live table with the status vocabulary, recent-run history, run now, pause/resume, install. (E /api/automations + actions)

5.6 Settings (D8; admin)

Section Content
Services & health services and capability areas with the vocabulary; re-check; run make check
Agent clients each client's config state (restart pending / in sync) and the servers connected to it; regenerate configs
Project connections the matrix across all projects (per-project toggles also live on each hub)
Routing & models the effective policy (shared + this machine's narrowing, §8); available routes and local models; probe providers
Tracing endpoint, UI URL template, and whether telemetry env vars are applied to launched runs (§10)
Code library source root, searchable categories, count of repos under the root
Research channels the arXiv feeds the scheduled poller reads, as a typed form (add, edit, pause, remove), and Poll now; a change applies on the next scheduled poll (#111; thin in v1, more in v2)
OmniRoute its status, link and enable/disable toggle (ported from the old OmniRoute page)
Checks run make check and see its output (ported from the old Checks page)
Security live checks where the server can make them; the rest marked "not checked"
First-run setup each capability area's state with a set-up action; how a fresh clone starts (D7)

6. New HTTP endpoints (default shapes)

All new endpoints live under /api/next/. Every POST goes through request_rejection (Host, Origin, token). Nothing takes a command, a path or an executable from the browser: runs refer to experiments, variants, models and routes by id, and the server resolves them (the same rule as dashboard_automations.start_run).

Method Path Purpose
GET /api/next/ui tabs, pinned items, last surface/item (§9)
POST /api/next/ui save the same
GET /api/next/policy effective routing policy and available routes for this machine
GET /api/next/runs run index rows; filters: kind, status, project, since
GET /api/next/runs/<id> one run: its events, checks and stand-in calls
GET /api/next/runs/summary spend today / 7d / 14d / month, broken down by model, surface and route
GET, POST /api/next/budget monthly cap; POST sets it
POST /api/next/ask start or continue a conversation: {session?, model, route, classification, chips[], text} → {run_id}
POST /api/next/approvals/<id> {decision: allow\|deny} for a pending permission prompt
GET /api/next/stream/<run_id> server-sent events: token, tool, approval, check, status, done, error
GET, POST /api/next/lab/experiments list / create
GET, PATCH /api/next/lab/experiments/<id> read / change setup, scope and classification
POST /api/next/lab/experiments/<id>/variants create a variant from another one
PUT /api/next/lab/experiments/<id>/variants/<v> save variant text
POST /api/next/lab/experiments/<id>/run run a selection of variants × models → {batch_id, run_ids}
POST /api/next/lab/experiments/<id>/promote {variant} → creates a branch and returns its name

Security acceptance for every agent-launching POST (ask, lab/.../run) (D6):

  • Refused with a foreign Host, a missing or foreign Origin, and a missing or wrong token. That's three tests per endpoint.
  • Refused when the requested route violates the policy for the classification (§8).
  • Refused when the ids don't resolve to server-side definitions.

Slice 5 security acceptance (epic #86, Part 2)

Slice 5 is not done until these exist and pass. Each test is first run against a version of the code that lacks the control, and must fail for the stated reason.

  • D6, per agent-launching endpoint: the three tests above (foreign Host, missing or foreign Origin, missing or wrong token), plus the route-policy and server-side-id refusals.
  • D17, routing: every known-bad test in §8.3 is refused by routing-check and again at run time, and the unknown-classification and unknown-route configs fail make check.
  • Tier enforcement, at launch (the epic's stance):
  • A reader session is launched with no write tools and no outgoing network, enforced by the tools granted at launch, never by the prompt. A launch request that adds a write tool to a reader is refused by the server.
  • The reader's launch arguments pass the strict-reader invariant the first red-team session used (phase2_command_problems in scripts/redteam_phase2.py, or an equivalent check for the launcher's own flags): built-in tools limited to the read set; every non-read tool of every attached MCP server denied by name, so a server that gains a tool does not widen a reader; the permission mode set explicitly, never inherited from the user's global mode; and only the settings sources the launcher intends. A test builds the arguments the launcher would use for a reader and runs them through that check; a launcher that drops a deny fails it.
  • A session cannot widen its own network allowlist. A reader that asks for extra hosts for a command is refused, and nothing reaches an unlisted host. Judged by what a local sink receives, not by the model's words.
  • A gated write in an actor session waits for the answer from POST /api/next/approvals/<id> (a request that passed request_rejection) and for nothing else: no classifier answers it, an unanswered prompt times out as deny (§7.2), and the write does not land first.
  • An actor session is never started with untrusted content in its input. Crossing from reader to actor goes through a summary the human reviews, and no endpoint skips that step.
  • Red-team re-run: the red-team session (#97, #98) is repeated against launcher-enforced tiers, with the same decoy-only tooling (make redteam-parity, the make redteam-p2-* loop), and its results are recorded as before. The first run targeted the Part 1 defences with the tier assumed failed, and the strict reader with a hand-written launch line. This one confirms the tiers themselves hold, and it must:
  • start sessions through the launcher, so the launcher's real arguments are what is attacked;
  • run the reader tier through the strict-reader attempts (the forbidden space, a write request, injection through each content channel, a read-scope sweep), with the transcript audit finding no call outside the tier;
  • exercise the approval path through the dashboard (approvals/<id>), with a person answering there. The first session's approval results are for Claude Code's own terminal prompts and do not carry over to a headless session with a permission-prompt tool;
  • cover each permission mode the launcher can run under.

A model's refusal stays inconclusive; the tier's result is what the transcript audit and the sink show.

The tier bullets above are refined by what the first red-team session could and could not show; the rest (the launcher's exact flags, the approvals tool's schema, §7.2's Verify) waits for the slice 5 issue.


7. Runner

7.1 Processes (D4, D13; defaults stated in D13)

  • One process per conversation or run:
  • agent mode: claude -p --output-format stream-json --input-format stream-json --include-partial-messages --permission-prompts host --permission-prompt-tool <approvals tool> --model <id>;
  • prompt mode: a direct HTTP call to the chosen route.
  • At most 3 runs at a time (a default; a setting). The rest queue in order.
  • The Ask panel uses your normal Claude Code configuration, so your hooks (including the write gate), skills and MCP servers apply.
  • A Lab agent-mode run gets a temporary CLAUDE_CONFIG_DIR built fresh for that run. It contains:
  • your hooks from ~/.claude/settings.json (so the gate still applies);
  • the variant under test as the only copy of that skill;
  • only the MCP servers the experiment names, with write tools pointed at the stand-in (§7.3).
  • Nothing else is inherited. The run's working directory is a temporary directory, isolated with the sandbox-exec helper from #84 where it applies. Both are removed after the run.
  • Telemetry variables are set on every launched process (§10). Verify: whether a parent trace context can be passed in.

7.2 Approvals (D12)

  • When claude -p asks for permission, the approvals tool holds the request.
  • The server records it on the run and pushes an approval event to the stream. The Ask panel shows Approve / Deny, and the answer goes back through POST /api/next/approvals/<id>.
  • Default: unanswered prompts time out as deny after 10 minutes.
  • Lab runs never prompt. Stand-ins answer instead.
  • Verify: the exact contract of --permission-prompt-tool, meaning the tool's input and output schema. claude --help (checked 2026-09-29) confirms the flags exist: --permission-prompts host|none, --permission-prompt-tool, --input-format stream-json, --resume.

7.3 Stand-in tools (D13)

  • standin_mcp.py is a local MCP server that registers the same tool names as the real servers the experiment uses (e.g. the AnyType write tools).
  • Every call is recorded (tool, arguments, time) in standin_calls, and a plausible success response is returned. Reads can be served from a small fixture set in the experiment.
  • Checks assert on these records.
  • Fallback, if stand-ins prove too limited (D13): a throwaway AnyType space created for a run batch and torn down after it. Verify first: whether the scoped API key may create and delete spaces.

7.4 Results and cost

  • Each run's final result (tokens, duration and, if reported, cost) is written to its run row. Verify: that the stream-json result message reports cost.
  • Where cost isn't reported (e.g. prompt mode against OmniRoute), it is computed from tokens and a per-model price table in tool-chest.toml, and marked "estimated".

8. Routing policy (D16, D17)

8.1 Shared policy, in tool-chest.toml (default syntax)

[routing.routes.anthropic]            # cloud, direct
kind = "anthropic"

[routing.routes.omniroute]            # cloud, via the local OmniRoute gateway
kind = "omniroute"

[routing.routes.local]                # Ollama on this machine
kind = "ollama"

[routing.policy.low_sensitive]
allow = ["*"]

[routing.policy.internal]
allow = ["local", "anthropic"]        # local + a named cloud list, starting with Anthropic direct only
confirm_cloud = "per_conversation"    # once per conversation or experiment

[routing.policy.private_sensitive]
allow = ["local"]                     # no cloud route, no override
  • The classification names are exactly tenancy.CLASSIFICATIONS.
  • A config that names an unknown classification or route fails make check (default: a new routing-check target).

8.2 Per-machine narrowing (D17)

  • An optional data_root/routing.local.toml may only remove routes from an allow list, or tighten confirm_cloud. Anything that would add a route is rejected at load time, with an error on Settings → Routing.
  • The same file marks the machine (machine = "personal" | "work"), which drives the promote confirmation (§5.4).

8.3 Deciding a run

  1. Classification = the action's explicit value, or else the current project's default. The domain decides that default: work → internal, otherwise low_sensitive (default mapping). Raising it is free; lowering it below the project default needs a confirmation.
  2. Route must be in the effective allow list, and for local routes, available on this machine now. If not, the request is refused with the reason.
  3. Cloud confirmation is recorded per conversation or experiment.
  4. Everything is logged: the run row records the classification, the route and the decision.

Known-bad tests (each must fail):

  • an internal run on a route not in the list;
  • a private_sensitive run on any cloud route;
  • a private_sensitive run on a machine with no suitable local model;
  • a per-machine file that tries to add a route.

9. State (D14)

9.1 Files

<data_root>/
  tool-chest.db                   SQLite: tables below
  routing.local.toml              optional per-machine narrowing (§8.2)
  lab/<experiment-id>/
    variants/<variant-id>/SKILL.md   plain files, diffable, editable outside the app
    fixtures/                        inputs and stand-in read fixtures
  • Decided: SQLite through Python's built-in sqlite3; variants as files; existing JSON/JSONL stores not migrated. (D14)
  • Moving other stores into one structured root is #83's job. This spec only adds.
  • Every row from Ask or the Lab carries its classification. None of it is ever indexed for search.

9.2 Tables (default schema)

Table Key columns
runs id, kind (ask/lab/automation), started_at, ended_at, status, project, classification, route, model, mode (agent/prompt), session_id, trace_id?, experiment_id?, variant_id?, batch_id?, tokens_in, tokens_out, cost_usd, cost_estimated, error?
run_events run_id, seq, at, type (token/tool/approval/check/status), payload (JSON; prompt text only if recorded)
approvals id, run_id, tool, input_summary, decision, decided_at
experiments id, name, artifact_kind, artifact_ref, project?, classification, models (JSON), checks (JSON), created_at
variants id, experiment_id, parent_id?, label, path, created_at
check_results run_id, check, result (pass/fail/notrun), evidence
standin_calls run_id, seq, tool, args (JSON), at
conversations session_id, title, project?, classification, started_at, last_run_id
budget month, cap_usd
ui_state key, value (JSON): tabs, pinned, last item per surface

Default: a schema_version pragma, and migrations as numbered SQL in scripts/migrations/, applied at server start.


10. Tracing (D15)

[tracing]
endpoint = "http://127.0.0.1:6006/v1/traces"   # default: local Phoenix; empty = no tracer
ui_url_template = ""                            # e.g. a trace URL with {trace_id} or {session_id}
record_prompts = false                          # only honoured when every configured backend is local
  • Write side: launched processes get:
  • CLAUDE_CODE_ENABLE_TELEMETRY=1
  • CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1 (span export is a beta)
  • OTEL_TRACES_EXPORTER=otlp
  • the endpoint and protocol
  • OTEL_LOG_USER_PROMPTS only when record_prompts is true and the endpoint is local. (Verified 2026-09-29, Claude Code monitoring docs.)
  • Route side:
  • No collector in v1.
  • An OpenTelemetry Collector is added only for a second backend or any cloud backend. It becomes mandatory for any cloud backend, which only receives spans with content stripped.
  • Read side: Traces, Costs and the budget read the run index only. The tracer is reached through ui_url_template.
  • Admission test for a backend: accepts OTLP over HTTP, and is self-hostable or has a free tier covering personal use.
  • Not built: vendor adapters, a cross-vendor query API, plugins.

11. Rollout (D18, amended 2026-10-12)

The new interface is built at /next; the old dashboard at / stays frozen (fixes only). There is one cut-over, at parity with the old dashboard, not after the last feature: when every section of the old dashboard has its new home, / serves the new shell and the old dashboard and the prototype are deleted. The model-running slices (5 to 8 below) land after the switch, inside the shell you already live in. Slices are small and independently testable; the list, the order and each slice's acceptance criteria are in epic #113. Every slice is built test-first and then run through /backpass.

Before the switch (v1, pain-first)

Group Slices Done when
Foundation S1: walking skeleton (shell frame, empty state per surface, server-side tabs on the new SQLite store with numbered migrations, Settings -> Services & health, browser-smoke CI job) the frame, the store and the browser test gate exist and are green in CI
Research management S2 feeds editor (#111), L2 staged review (#112), both thin a feed can be edited and a staged entry promoted or deleted from the UI
Settings S3 Agent clients, S4 Project connections, S5 Code library, S6 First-run setup, S7 Security, S8 OmniRoute, S9 make check each replaces its old page
Library L1 Find and Read, L3 Skills and MCP, L4 freshness and update, L5 Ingest repos, L6 Ingest proposals each replaces its old section; absorbs #54, #53
Today T1 Needs attention, T2 Commitments, T3 Where you left off, T4 Inbox, T5 Glance column the home surface; run widgets show empty states until slice 5
Projects P1 index and New project, P2 hub header, Repository and connections, P3 hub PKM, Commitments and patterns, P4 hub Lab and runs (empty states) absorbs #49, #50
Parity extractions X1 ⌘K palette, X2 Capture (from the Ask panel slice), X3 Automations page (from Agents), X5 Harnesses catalog nothing in the old dashboard is left without a home
Cut-over X4 switch over, delete the old dashboard and the prototype every old section used and verified in its new home; closes #57

After the switch (each gets its own grill; their Defaults depend on V1-V8)

# Slice Done when Absorbs
5 Run infrastructure runner, SSE, run index tables (new migrations), routing policy + routing-check + known-bad tests, approvals, telemetry env, the §6 security tests
6 Ask panel the full §4.2 (capture already shipped as X2) #48, #51, #52 (capture attachments)
7 Agents traces, costs + budget, the tracer link (automations already shipped as X3)
8 Lab experiments, CodeMirror variants, both modes, stand-in tools, checks, promote, budget hold candidate run log for #67

The security epic and red-team session (#86) are done; slice 5 is the first slice that launches agents, and no agent-launching endpoint ships before its security tests (§6, "Slice 5 security acceptance"). Every slice passes make check and its own tests, and changes no existing endpoint's behaviour unless its issue says so.


12. Out of scope (v1)

  • Ambient or background generation (D2). A native wrapper, a system-wide hotkey, a menubar (D6). Access from other devices, and sync between machines (D8, D16).
  • Costs for Claude Code sessions not launched by the dashboard (D15).
  • 67's wider harness lab beyond the run index.

  • Research channels beyond management: the ingestion pipeline is built (#80). Its management UI is in v1, kept thin (S2 feeds editor, L2 staged review); anything beyond that (per-feed run history, rule-noise views, query builder) is v2.
  • A light theme (a later slice, §3.4).

13. Verify before building

# Unknown Needed by
V1 The input/output contract of --permission-prompt-tool slice 5
V2 Whether the stream-json result message reports cost slice 5
V3 Whether Claude Code accepts a parent trace context from the launching process (otherwise link by session_id) slice 5
V4 How agent mode reaches non-Claude models: Claude Code's model mapping (behavesAs, deferred by Nathan), and for Ollama whether it goes directly or via OmniRoute slice 8
V5 Whether the scoped AnyType key may create and delete spaces (the D13 fallback) only if the fallback is needed
V6 Whether Phoenix stores to SQLite by default when run as one process slice 7
V7 That the CLAUDE_CONFIG_DIR composition in §7.1 loads exactly the listed hooks, skill and servers — calibrated on a known case first slice 8
V8 Per-provider agent-mode support through OmniRoute: the three-step probe on providers that answered (so far cfp plus the Claude routes) slice 8

14. For Nathan, not decided by tool-chest

  • Whether employment or client agreements allow internal material to go to Anthropic, and whether skills written on a work machine may be pushed to this public repo (D17; same kind of question as A5). Until then: internal goes to named routes only, and promote on a work machine needs a confirmation.
  • AnyType syncs every space in the account to every device, including an employer laptop (D17).