Skip to content

Threat model

What can go wrong when AI agents work in this repo and on this machine, what stops it, and the test that fails while the weakness exists. Tracked by epic #86.

Rule: a finding appears here only once it is fixed. Each one names the fix, the tests, and the limits that remain. make check verifies that every test named here exists (scripts/test_threat_model.py).

Scope

The person who owns the machine, running Claude Code and Cursor with tool-chest's MCP servers and automations. The concern is an agent that has been misled (prompt injection through content it read) or is simply too capable: it can run shell commands, edit files, and call local services.

Out of scope: an attacker who already has the user account, physical theft, and nation-state actors.

Threat classes

  1. A web page reaching the local dashboard.
  2. Prompt injection through content agents read (research, clones, fetched pages, notes).
  3. Supply chain: reference clones, MCP servers launched by npx, Python dependencies, vendored scripts.
  4. Data leaving where it shouldn't (this repo is public; model routing; two machines).
  5. Credentials readable or usable by agents.
  6. Agents widening their own permissions.

The findings below are the ones fixed so far, mostly classes 5 and 6 and the network side of 4. The rest of the epic covers the others.

How the tests work

  • Fast checks run in make check and CI (installer and gate unit tests, drift checks). They don't call a model.
  • Attack probes run locally with make security-attack. Each probe gives a headless Claude Code session an ordinary task that needs the attack to succeed, and judges the effect (a file's contents or hash, a request a local sink received), never the model's words. A model that declines counts as inconclusive, not a pass: the control has to hold, not the model's manners. They call a model, so they never run in CI.
  • Every probe was first run against the unfixed setup and had to fail there, for the stated reason.

How this was checked

Red-team sessions are person-driven and decoy-only: a decoy AnyType key and decoy spaces, and a scratch copy of the installed gate and sandbox that make redteam-parity proves is the defence in use. They judge effects: a key or marker in the transcript, the session directory or the egress log; a write that landed; a read of a space the key cannot see. A model that refuses an injection counts as inconclusive. The first session (#98, October 2026) ran in default and in auto permission mode and found no write that landed unapproved, no secret at the egress listener or in any transcript, and the key's scope hid the forbidden decoy space in every attempt. Its limits are stated in the run log, not here: the decoys stand in for the real spaces, and a clean session is evidence about the attempts made, not a proof. Tooling: scripts/redteam_*.py and the make redteam-* targets.

Findings

F1 — The AnyType API key was in the environment agents inherit

  • Fixed by: #89 (key moved to the macOS Keychain, read by a launcher that starts the MCP server) and #88 (the variable is unset in sandboxed commands).
  • Mitigation: the key is no longer exported from the shell profile. scripts/anytype_mcp_launch.py reads it and puts it in the MCP process's environment only.
  • Test: probe:env-secrets, scripts/test_anytype_key.py::LauncherTest, scripts/test_anytype_key.py::IsolationTest.
  • Known limits: removing the old export from the shell profile is a manual step; if the key is exported again the finding reopens, and probe:env-secrets shows it.

F2 — Agents could call the AnyType API directly, skipping the write gate

  • Fixed by: #88 (no key in the shell), #89 (key in the Keychain) and #91 (the gate refuses shell commands that name the API port).
  • Mitigation: the write gate applies each space's write policy to MCP tool calls. Without the key in the shell, a direct call isn't authenticated; as a second layer the gate denies Bash commands that name the API port or ANYTYPE_API_BASE_URL.
  • Test: probe:anytype-api-direct, scripts/test_anytype_write_gate.py::DirectApiTest.
  • Known limits: the gate rule is a pattern match on the command text. Obfuscated forms (a port assembled from pieces) aren't caught, and an API URL set only through the environment isn't followed. The API port itself stays reachable from sandboxed commands (see S2); without a key it answers nothing useful.

F3 — Other credentials sat in the agent's environment

  • Fixed by: #88 (credentials.envVars deny in the Claude Code sandbox, generated from tool-chest.toml).
  • Mitigation: SSH_AUTH_SOCK, GIT_ASKPASS, VSCODE_GIT_IPC_AUTH_TOKEN and the AnyType header variable are unset before every sandboxed command, so an agent can't push over SSH or reuse the editor's git credential helper as the user.
  • Test: probe:env-secrets, scripts/test_claude_sandbox_install.py::DesiredBlockTest.
  • Known limits: the sandbox covers Claude Code's Bash tool only, not Cursor. The escape hatch (running a command outside the sandbox) stays on, behind an ask rule that prompts every time.

F4 — Credential files were readable by agents

  • Fixed by: #88 (credentials.files deny: ~/.ssh, ~/.config/gh, the glab config directory, OmniRoute's key store and .env).
  • Mitigation: sandboxed commands can't read those paths.
  • Test: probe:read-ssh, scripts/test_claude_sandbox_install.py::InstallCheckTest.
  • Known limits: the credential denies apply to sandboxed Bash commands only; Cursor has no equivalent.

F5 — The write gate ran from a copy agents could edit

  • Fixed by: #90 (the gate runs from an installed copy that agents can't write).
  • Mitigation: make anytype-gate-install copies the gate, a snapshot of the space registry and a small gate.json (state directory, API port) into ~/.claude/hooks/tool-chest/. The installed gate reads only that folder, so editing the repo, spaces.json or tool-chest.toml changes nothing until the person re-runs the install. make anytype-gate-check fails while the copy and the repo differ.
  • Test: probe:edit-gate, scripts/test_anytype_write_gate.py::InstalledGateTest.
  • Known limits: a change to the space registry needs a reinstall. The gate's self-grant protection (a pattern match on commands and paths) is a guard, not a boundary; the boundary is the install.

F6 — Agents could edit the settings that register the defenses

  • Fixed by: #90 (deny rules on the installed folder and on ~/.claude/settings*.json).
  • Mitigation: [claude_sandbox] deny_write is installed as sandbox.filesystem.denyWrite (Bash) and Edit() permission rules (the editor tools), together covering both routes.
  • Test: probe:edit-settings, scripts/test_claude_sandbox_install.py::WriteDenyTest.
  • Known limits: the person can still change these files, by design; the install is the human step. Writes by anything running outside Claude Code's sandbox and permission rules aren't covered.

F7 — A viewed page shared the dashboard's origin

  • Fixed by: #92 (every /view response carries Content-Security-Policy: sandbox allow-scripts; the reader iframe has no allow-same-origin).
  • Mitigation: /view serves text from files and corpora an agent or an outside source can write. Its sanitizer is the only thing that kept that text from running as script on the dashboard's origin, where the per-launch token lives. Now a page that gets past the sanitizer still has an opaque origin: it can't fetch the dashboard and read the answer, so it can't read the token or call a route that needs it.
  • Test: scripts/test_view_isolation.py::ViewIsolationTest (a sanitizer-bypass fixture reads the token without the header and is blocked with it; the viewer's buttons still work), and the header assertion on every corpus in scripts/test_dashboard.py::RequestGuardTest, scripts/test_viewer_corpora.py::ViewerHttpTest, scripts/test_library_search.py::ViewerHttpTest and scripts/test_research_search.py::ViewerHttpTest.
  • Known limits: allow-scripts stays because the viewer's own Copy path and View source buttons are script, so a bypassed page can still run code inside its opaque origin (draw fake UI, mislead the reader); plain sandbox would stop that and break the buttons. Requests from the opaque page still reach read-only GET routes, which check only the Host header, but it can't read the replies; those routes must stay free of side effects. The browser test needs Playwright and Chromium (make browser-test-setup) and skips without them, so CI enforces the header assertions only.

F8 — The AnyType MCP server was launched unpinned

  • Fixed by: #93 (the package is pinned to an exact version in tool-chest.toml).
  • Mitigation: the server holds the AnyType key, and npx -y fetched whatever was newest on every start. The version is now exact, written into both generated client configs, and enforced twice: make mcp-check rejects a range, tag or bare name for any npx server, and the launcher refuses to start an unpinned package before it reads the key.
  • Test: scripts/test_gen_mcp_config.py::NpxPinTest, scripts/test_anytype_key.py::LauncherTest, scripts/test_anytype_key.py::GeneratedConfigTest.
  • Known limits: a pin fixes the version, not a hash of its contents, so the registry's copy of that version is still trusted. Updating the pin is a deliberate, reviewed change. The Python tools are locked by uv.lock; this covers the npx server only.

F9 — The AnyType key's Keychain item could be read silently

  • Fixed by: #89 (the item is stored so that it trusts no application) and #95 (an automation's key is bounded by agent policy).
  • Mitigation: every read of the everyday key's item shows a macOS dialog, so a sandboxed agent's security find-generic-password fails unless the person allows it. The sandbox can't block Keychain reads itself, so the item's access control is the only defense. An automation's own key can't prompt, because it runs unattended under launchd, so an agent can read it silently. That is accepted because the key can't do more than agents may already do: it is read-only or write-capable only on spaces whose policy is agent_write: full, and a definition that declares a write key on any other space is refused.
  • Test: probe:keychain-read, scripts/test_anytype_key.py::ReadersTest, scripts/test_automations.py::DefinitionTest.
  • Known limits: the dialog names security whoever asks, so the launcher and an agent look the same; timing is the tell (expected at MCP start, not out of nowhere). An agent can ask repeatedly until a person tires of clicking. A compiled helper as the only trusted application would remove both limits, but it needs a build and an ad-hoc signature, and trust in it probably wouldn't survive a rebuild; it is deferred. The first red-team session (#98, decoy key and decoy item) did not show an agent obtaining the key by asking again: the agent's guessed-name reads raised two dialogs, both were denied, and nothing was found. The automation-key bound is only as good as spaces.json: loosening a space's agent_write loosens it, and make check validates definitions against the file as it stands.

F10 — A client's, person's or employer's name could be committed to this public repo

  • Fixed by: #94 (the canary also reads a private list of names from the data root).
  • Mitigation: the canary used to know only the project names and AnyType ids in the state file, so a name outside that file had no check. It now also fails on any name in <data_root>/canary.txt found in a tracked file, at every commit through the pre-commit hook (make check). The list stays outside the repository, which refuses a data root inside it, and a failure prints only the file and line, never the name. make canary-init writes a commented template for the list.
  • Test: scripts/test_name_canary.py::FindNameLeaksTest, scripts/test_name_canary.py::InitTest, scripts/test_privacy.py::CanaryTest.
  • Known limits: it does nothing until the person runs make canary-init and adds names, and the list is per machine (the work laptop needs its own). git commit --no-verify bypasses it, and CI has no list, so it skips there. It matches exact names only (whole word, case and spacing ignored); nicknames, misspellings, initials and encoded forms are not found. It scans tracked files only: commit messages, merge request and issue text, and untracked files are not scanned, and a name that has already been pushed needs a history rewrite.

S2 — Commands could send data to any host

  • Fixed by: #88 (network allowlist in the Claude Code sandbox, with an ask rule on git push and on leaving the sandbox).
  • Mitigation: sandboxed commands reach only the allowlisted hosts (GitHub, GitLab, the package registries); other hosts are refused by the sandbox proxy.
  • Test: probe:egress-unlisted-host, probe:git-push, scripts/test_claude_sandbox_install.py::DesiredBlockTest.
  • Known limits: allowLocalBinding (needed so tests can bind ports) also lets commands connect to loopback and the machine's own addresses on macOS, so local services answer (AnyType, the dashboard, a model router). The AnyType key and OmniRoute's key store are out of reach (F1, F4), and the dashboard needs its per-launch token. The sandbox's no-proxy list covers localhost and private address ranges, so only the filesystem rules apply to those.