Marker without trail
“Something went wrong here.” Strong human intent, weak reproduction context.
Human intent is the missing signal
A human-anchored pattern for agent debugging: pair timestamped human intent with an ordered semantic trail, then let an agent draft a reproducible scenario without guessing what mattered.
The lifecycle
A useful recorder does not capture everything and hope an AI finds the problem. It gives a tester a precise join point between their judgment and the machine-observed path around it.
Start a bounded session with a clear clock origin and only the metadata needed for reproduction.
Drop a bug, note, or moment with a relative timestamp and short human explanation.
Retain coarse screens, semantic actions, selected failures, and errors—not raw secrets.
Stop capture, inspect the session, then explicitly submit or discard it.
Validate metadata, markers, and trail events; merge them into deterministic order.
Ask an agent for preconditions, steps, expected behavior, reported observation, and assertions.
Text equivalent: a permitted human records context, marks a significant instant, preserves the preceding trail, reviews the capture, exports a typed timeline, and gives it to an agent for a reviewable draft.
Two signals
A marker says why this moment deserves attention. A trail says how the tester arrived there.
“Something went wrong here.” Strong human intent, weak reproduction context.
“Many things happened.” Strong chronology, weak understanding of what mattered.
Evidence layers
This pattern is grounded in a working implementation. The recommendations below generalize its strongest boundaries without claiming that any single signal proves causality.
Defines the container: platform, timing, lifecycle, build label, and optional test intent.
Does not prove: that a defect occurred.
Captures what the tester considered significant and what they reported at that instant.
Does not prove: cause, correctness, or completeness.
Provides bounded, semantic breadcrumbs such as screens, stable action labels, and selected failures.
Does not prove: that uncaptured events never happened.
Adds access-controlled visual context when available. The structured envelope still stands without it.
Does not prove: durable state or root cause.
Fictional example
All actors, screens, labels, times, and references below are placeholders. The example separates recorded evidence from inference.
actor: <tester-alias>build: <build-label>replay: <optional-replay-reference>
Provenance: marker at 00:34; supporting trail window 00:12–00:33.
Trust boundaries
The client captures untrusted events. The server enforces policy. The export carries reviewed evidence. The agent produces an interpretation that still needs independent validation.
Text equivalent: human intent flows through an untrusted client into a server policy boundary; reviewed records become a typed export, an agent drafts a hypothesis, and a person or test runner verifies it. Visual replay is an optional side reference.
Failure modes
“Structured” does not automatically mean safe, ordered, complete, or true.
Deny capture on sensitive screens. Never retain raw field values, request bodies, tokens, or full URLs. Redact before agent access and define retention separately.
Prefer stable semantic identifiers, deduplicate repetition, cap volume, and show a bounded window around each marker.
Use monotonic offsets plus a per-session sequence and stable secondary ordering. Relative time alone cannot settle ties.
Restore active state on reload, permit one live session per actor, and close abandoned sessions with an explicit reason—never silent submission.
Show pending versus synced state, acknowledge final flushes, use idempotency where needed, and represent known capture gaps.
Label testimony, recorded facts, and inference separately. Preserve provenance. Never present a draft scenario as root-cause proof.
Adoption ladder
The smallest valuable version is marker-only recording with explicit review. Rich capture can arrive later.
Dedicated capability, owner-scoped sessions, one active session.
Exit gate: bypassing the UI still cannot bypass authorization.
Timestamped bug, note, and moment testimony through an explicit lifecycle.
Exit gate: reload preserves recording or review state.
Screen transitions and stable action labels without sensitive values.
Exit gate: markers show a concise ordered context window.
Typed metadata, markers, and trail with deterministic ordering.
Exit gate: malformed or foreign-session exports fail closed.
Preconditions, steps, reported observation, assertions, and provenance.
Exit gate: inference cannot be mistaken for executed proof.
Visual context, bounded inspection, retention controls, and redaction checks.
Exit gate: the core workflow still works without replay.