Test suite

Every tenant accumulates a regression suite from its own approved captures. Schema changes are gated on it. This is what makes agent-authored extraction safe rather than merely possible: you can edit a schema and prove nothing broke.

Cases

A case is a stored message, a bucket, and the field values expected from it. Cases come from normal use: promote_to_test on a capture takes its current values, which are what a person approved or corrected. Nobody authors a case by hand, though you may pass expected to override or narrow the fields checked.

Prefer tricky cases: anything that needed a correction, a task, or a repair. A suite of easy cases proves nothing. A case promoted from a corrected or held capture is marked tricky automatically, and tricky cases survive when a bucket reaches its cap (200 cases; the oldest easy case makes room).

Cases pin their message. It stays through retention purges for as long as the case is live, is included in tenant export and delete, and is visible to nobody outside the tenant. A sandbox's expiry still deletes everything.

Running

run_tests runs a bucket's cases against its current schema, or against a candidate schema you pass, and changes nothing. Failures come back with each field's expected and actual value; get_test_run has the full per-field result. Runs are kept 90 days.

The gate

update_bucket with a schema runs the bucket's suite first. All pass: the change applies and the result carries test_run. Any fail: the change is refused with the failures, a config_change task records the attempt, and the bucket is unchanged. Fix the extractors, retire the cases that are wrong with remove_test, or pass force: true with the configure scope.

The tenant setting require_passing_tests (also per bucket) is on by default. A bucket with no cases has no gate.

Workflow for an agent

  1. Ingest real messages. Read what came out with get_capture.
  2. Correct what is wrong with correct_field; approve.
  3. promote_to_test the ones that needed work.
  4. Iterate on the schema with dry_run_schema and run_tests with a candidate.
  5. update_bucket. The suite runs; the change applies if it passes.

This page as Markdown.