Local telemetry

Status: source candidate today for the Node diagnostics stream described under "Today" only; everything else is planned (T0–T7). Decisions: local telemetry design, defaults per release channel.

Goal

Observe and improve all of Maia (Nodes, harness Sessions, Skills, orchestration Runs, clients and CI) from data that stays on the user's machines. Each Node keeps its own history; nothing leaves a machine unless a person exports it. Telemetry feeds reports that people and Skills read; it never changes Skills, routing or configuration by itself.

Today

• Only Maia Node emits telemetry: Effect logs, spans and metric snapshots as NDJSON records (maia.observability/v1alpha1), written to stderr and to a rotating events.ndjson file of at most 5 × 5 MiB in the profile's observability directory. Nothing reads the file back; no protocol operation, CLI command or client uses it.
• At shutdown the Node gives pending file writes 500 ms. A write still pending keeps the profile lock. After node.shutdown the process waits for that write and then exits 0. A failed run exits with its code, and a signal or a closed Desktop pipe exits 0; both leave the lock for the next Node to recover. The run loses the pending record and every record queued after it, and a write cut off mid-record can leave a partial line. Nothing reads the file back, so the loss costs only diagnostics.
• domain_events in maia.sqlite and the observe.events stream carry the authoritative journal; session.observe carries live activity and process usage.
• No client, CI or harness telemetry is collected. Harness launch environments pass only an allowlist of variables, so every OTEL_* variable is dropped, and Maia's managed Pi worker bundles a no-op OpenTelemetry shim.
• Turn token usage is stored for Claude Code and Codex on completed turns (turn.show, turn.completed). Cost, model and duration are not stored, and Pi turns carry no usage (T1). Claude's total_cost_usd is cumulative per Session, so a turn's cost is a difference.

Design

The Node owns telemetry in services/node/src/telemetry/ (decision). It is history, not authority: the control store (maia.sqlite) stays the source of truth, and telemetry lives in its own telemetry.sqlite per profile, so high-volume writes, retention and purges never touch authoritative state. Both use the node:sqlite engine the Node already uses; no new dependency.

Records follow a protocol-owned maia.telemetry/v1 schema with closed kinds; attributes are allowlisted (at most 4 KiB per record). Harness identity attributes (user.email, account and organization identifiers) are dropped on ingest, and attributes a harness supplies about its Session are not trusted: the Node stamps identity from the bearer token.

Levels, per profile; a Project may only narrow them:

Nightly and development builds collect basic and detailed so the people building Maia see everything Maia measures; the data still never leaves the machine. No level keeps conversation content: prompts, responses and tool output stay in the harness's own files (D-168). Their first run says so plainly and shows how to lower the level (maia telemetry level).

Export is a capability, off by default on every channel, with preview, confirmation and receipt for one destination: an OTLP-JSON-lines file that standard tools read, or a push to an endpoint the user names. Another user's data reaches you only as an exported, consented aggregate.

Retention: diagnostics 7 days and at most 25 MiB; 1-minute metrics 7 days, then hourly for 90 days; events 30 days; CI 90 days; the whole store capped at 1 GiB, oldest first, each eviction recorded.

Reads: telemetry.status, telemetry.query over named views (usage, skills, runs, harness, node, ci, regressions) with closed group-by fields, at most 90 days and 500 rows, each result stating its coverage and gaps; telemetry.level, telemetry.purge and telemetry.export with preview and confirmation. The CLI family is maia telemetry; MCP exposes the two reads, scoped by Project grant, so a Skill can read its own statistics. A control Node queries remote Nodes through the same operation and merges aggregates; no content crosses Nodes.

Improvement loops are reports: Skill success, tokens, cost and retries; success and cost per harness and model; capacity from host samples and account windows; regressions against a baseline. Acting on them is a Skill's or a person's decision.

Harness settings Maia injects at the detailed level: for Claude Code, CLAUDE_CODE_ENABLE_TELEMETRY=1 with OTLP HTTP/JSON metrics and logs to the Node's loopback endpoint and a per-Session token; for Codex, [analytics] enabled = false (its default metrics go to OpenAI) and OTLP HTTP/JSON logs and traces to the Node in the Maia-managed CODEX_HOME; for Pi, an adapter on its TelemetryContext over the existing worker boundary. Content flags stay unset.

Constraints: no rows per token; no Project, Session or user ids as metric labels; no raw SQL for clients; OTLP over HTTP/JSON only (no protobuf or gRPC); no implicit reads of harness transcript directories. Also out of scope: OpenTelemetry SDKs or a Collector, analytics databases inside Maia, remote telemetry by default and automatic changes driven by telemetry (decision).

Known issues to fix first (T0)

• the schema string maia.observability/v1alpha1 is written in four places;
• two stacked drop queues report saturation with different messages;
• the physical batch writer orchestrates with Promises beyond its boundary;
• every record is written twice (stderr and file);
• process-resource metric descriptors are declared at module scope, so they can miss later telemetry registries;
• every named span is written; nothing is sampled.