LLM-API.md — the LLM contour: architecture and contracts

Audience. Engineers and operators working on the platform. This is an

internal architecture/contract document, deliberately not a page in

DocsController::PAGES (the same treatment BILLING.md, EMAIL.md and

STRIPE-PLAN.md get): it describes internals, not a consumable HTTP

surface. The public, agent-facing documentation of the HTTP API lives in

docs/AGENT_API.md §15, which is served at /docs/agent-swarm, and

the machine-discovery artefacts (/llms.txt, /agents.json,

/api/v1/openapi.json, /api/v1/errors) are generated from the router.

ADR-072 records the decision and its cost (a public page would need a nav

slug + a title key in all 49 active locales + the C7 ×3 sync, for no new

public surface).

Operations (keys, dashboard, purge policy, owner smoke, failure playbook):

docs/ADMIN-LLM.md. Environment keys: docs/ENVIRONMENT.md.

Error codes: docs/ERRORS.md.

---

1. Layers

Direction of dependence is one-way. Controllers → domain services → LlmClient → transport. Nothing below LlmClient knows about HTTP requests, sessions, agents or admin users; nothing above it may open a socket to a provider. That single rule is what makes "keys are invisible" provable rather than aspirational.

2. The provider catalog

Six slugs, frozen (LlmClient::PROVIDERS): bothub, openrouter, deepseek, qwen (DashScope compatible-mode), chatgpt (OpenAI), claude (Anthropic). Five speak the openai dialect, claude speaks anthropic. An unknown slug is a 422 schema_invalid carrying the frozen six-slug vocabulary — the API never guesses.

Resolution order for a key: **DB ciphertext row first, then LLM_<SLUG>_API_KEY**. Neither present ⇒ llm_not_configured (503) with zero outbound requests.

3. The cache contract (mandatory, not an optimisation)

The cache is a requirement of the design, not a performance afterthought: two identical requests must not cost twice. Contracts, all frozen by LlmCacheTest / LlmCacheAdminTest:

• Key = `sha256(VERSION ␟ provider ␟ model ␟ framed-messages ␟

significant-options), using \x1f` unit separators so no printable payload can forge a boundary.

• Message framing is length-prefixed per field

(mb_strlen(role):role|mb_strlen(content):content \x1e), so {system,user} can never collide with {user,user} carrying the same bytes, and a role word embedded in content cannot fake a message boundary.

• Normalization (whitespace collapse + trim) applies only to message

text. Identifiers participate verbatim: gpt-4.1 ≠ gpt41.

• Every model-significant option participates (ksort + JSON). Only the

transport/ledger/secret blacklist is dropped, and it fails safe: an option added later changes keys ⇒ misses, never collisions.

• Identity never fragments the cache. agent_id, purpose, timeout,

retries, cache_ttl and anything auth-shaped (api_key, authorization, key, token, headers, transport) are in NON_KEY_OPTIONS. Two agents asking the same question share one row; a caller passing a key in options can neither split nor poison a key.

• VERSION bump ⇒ every old row becomes a miss. The version is part of the

hashed material and rows are not rewritten, so the safe failure mode is "cache cold", never "cache wrong".

• Failed responses are never cached. isCacheable() is binary (empty

content, an error envelope, ok:false, Anthropic type:"error", finish_reason: error|content_filter) and is enforced both inside the cache and at the call site. A cached failure is worse than a miss: the provider recovers in seconds, the bad answer stays for the whole TTL.

• store() is best-effort — any storage fault degrades to

['stored' => false] plus a warning log and never throws.

• lookup() returns null on miss / disabled / expired / corrupt row.
• short_key = sha256(model|normalized last user message) is

display-only (admin tooltips, log lines). The full key is the collision authority.

• Keys are never part of the key or the value. Not the cache key, not

response_json, not the ledger.

TTL: LLM_CACHE_TTL (default 86400 s), per-provider override LLM_CACHE_TTL_<SLUG>, both clamped to 60…2592000.

4. Purging

Three policies, one shared door (LlmAdminService::purge() → LlmCache), two surfaces: the admin button (expired|all, whitelist frozen) and the console (all three). Full operator detail, markers and suggested policy: docs/ADMIN-LLM.md §5.

purgeOlderThan(int $seconds, ?string $now = null) deletes by age (created_at), which is orthogonal to TTL:

• the boundary is strict — a row exactly $seconds old survives;
• expires_at is irrelevant, so NULL-TTL ("immortal") rows are removed by

age while --expired will never touch them;

• $seconds < 1 throws llm_purge_invalid_age rather than being read as

"delete everything";

• $now is an explicit test seam (the same honesty seam as the

constructor's $version), which is how the exact boundary is asserted instead of raced. Without it the CLI's cutoff comes from the wall clock at second granularity and an exactly-N-seconds-old row is a coin flip.

Hot-path cache methods degrade silently; purge methods throw. An operator action that silently no-ops would lie.

5. The usage ledger: NULL versus zero

llm_usage records one row per chat() call: provider, model, tokens, cost, cache_hit, latency_ms. The token and cost columns are nullable: a provider that does not report usage yields NULL, and NULL survives the cache round-trip (the columns are copied as-is from ChatResponse).

Consequences that must not be "cleaned up":

• SUM() ignores NULL by design, so roll-ups stay correct.
• hit_rate is null until the first call exists — no data is not 0 %.
• A NULL cost means "the provider did not say", never "free".

6. Task lifecycle

llm_tasks carries pending → running → done | error, with result_json on success and a typed {code, message} error on failure; finished_at is set on both terminal states.

Execution is synchronous inside the create request. This is an honest minimum, not an oversight: there is no llm worker infrastructure, so rather than fake a queue the create call runs the work and persists the terminal state before responding. Two properties follow and are pinned by tests:

• Polling is free. GET /api/v1/llm/tasks/{id} is a pure re-read; repeated

GETs never call the provider again.

• Crash-stranded rows stay visible. If the process dies mid-run the row

remains running, and idx_llm_tasks_status keeps both a future async poller and ops visibility of those rows possible without a schema change.

Ownership: a task belonging to another agent is **indistinguishable from a nonexistent one** (404 llm_task_not_found). agent_id NULL is reserved for a future admin/CLI path — the HTTP surface always authenticates an agent token.

7. Reliability

• Retry/backoff around the transport, bounded by max_attempts;

llm_unreachable is raised only after they are exhausted.

• Circuit breaker: repeated failures open it, and while open calls fail

fast with llm_breaker_open (503) without touching the network.

• Fail-soft ledger: writing llm_usage must never fail a user-visible

request.

• Exception → public status mapping (frozen): llm_not_configured 503,

llm_unreachable 502, llm_api_error 502 (with details.http_status), llm_invalid_response 502, llm_breaker_open 503. Note the details key is http_status, not Stripe's status — the block-2 client froze it first and the docs were aligned to the code.

8. The agent surface (block 4)

Swarm zone only (agent-swarm.iya.ru), authenticated with swarm agent tokens (primary or scoped):

Design rules with reasons:

• Money paths are POST-only. A crawler or prefetch GET must never be able

to trigger a paid provider call, so GET /api/v1/llm/chat answers 405.

• Two kill switches, checked before auth and DB work: SWARM_ENABLED

(swarm_disabled) and LLM_API_ENABLED (llm_disabled).

• Scope separation: a read-only token cannot spend

(insufficient_scope) but can poll.

• Session auth was rejected for the owner/PO path: COOKIE_DOMAIN is

host-only by default (an apex session never reaches the swarm host) and /api/ is CSRF-exempt, so a session-authenticated POST there would be a CSRF hole. The PO delegates through their own agent credential instead (ADR-047 style). See ADR-072 for the rejected alternatives.

• Envelope: {ok, data, meta} / `{ok:false, error{code, message, details,

request_id}}, with a format=html` twin.

9. Security invariants (frozen by tests, not by convention)

1. Keys exist in exactly one place at runtime: a transport header built inside

LlmClient. They are absent from logs, exception messages, cache keys, cache values, the ledger, llm_tasks, swarm_audit, every HTTP response body, and the CLI's stdout.

1. Audit carries flags only. swarm_audit rows for llm.* record

provider/model/message-count/prompt-bytes/result-code/task-status — never message or prompt contents. A canary test asserts a planted message body never reaches the audit table.

1. Admin reads are mask-only: no plaintext, no ciphertext, ever.
2. Robustness surfaces are review-clean: the global Disallow: /api/ in

robots.txt already covers /api/v1/llm/, no HTML pages were added (sitemap untouched), and the public counters are DB statistics rather than route counts (untouched).

10. Configuration and parity

Seven registered keys (LLM_CACHE_ENABLED, LLM_CACHE_TTL, LLM_API_ENABLED, RATE_LIMIT_SWARM_LLM_CHAT, RATE_LIMIT_SWARM_LLM_TASK, LLM_MAX_PROMPT_BYTES, LLM_CHAT_MAX_MESSAGES) live in Config::defaults() and .env.example — the B-005 contract, because a key absent from defaults() silently ignores the environment. Provider keys are secretKeys and are not committed. Block 5 added no new config key and no new i18n key (the console is not localized; the keyset stays 1875 with locales frozen at 49).

11. Test and drill map

12. References

docs/ADMIN-LLM.md · docs/AGENT_API.md §15 · docs/ENVIRONMENT.md · docs/ERRORS.md · docs/BILLING.md (the same fake-sink honesty pattern) · ADR-072 (decisions + rejected alternatives), ADR-047 (agent tokens), ADR-058.4 (plan multiplier scales the limit only), ADR-043e/047.7/058 (the C7 surface ritual) — all in REWORK-STATE.md.