Understanding

Observability

Solves: understanding what an agent did and why, after the fact.

Emerging — seen across multiple products; the shape is still settling.

watching →emerging →established →contested →fading 177 sightings since Aug 2026
Ask an assistant about this pattern Ask ChatGPT ↗Ask Claude ↗

Observability helps humans understand an agent's work: what it did, what shaped it, where it is now, and what came of it. Logs, traces and audit trails are the evidence underneath. The experience is the understanding they make possible.

When to Use It

  • Humans need to understand an agent's behavior after the fact
  • Debugging, compliance, or audit needs a record
  • Traces feed evaluation and improvement

When Not To

  • Logs would capture sensitive data with no retention plan
  • Nobody will read raw traces; build summaries on top
  • Understanding is standing in for limits the product should set on what the agent may do

Three views

The exchange between the human, the agent and the system the agent acts on; who holds each part of it; and the component that implements it.

01 · the interaction
human agent system trace · timing · cost every call, write and decision goes to the trace the trace, with timing and cost what it did and why, step by step asks why a step happened answers from the trace
02 · who holds what
human interface agent Surface: Where the exchange appears — inline, panel, or notification 01 Surface Trigger: What starts it — the agent proposes, or the human initiates 02 Trigger Decision: The person decides: accept, redirect, or stop 03 Decision Record: What persists afterward — receipt, log, or memory entry 04 Record
human interface agent Surface: Where the exchange appears — inline, panel, or notification 01 Surface Trigger: What starts it — the agent proposes, or the human initiates 02 Trigger Decision: The person decides: accept, redirect, or stop 03 Decision Record: What persists afterward — receipt, log, or memory entry 04 Record
  1. 01 Surface Where the exchange appears — inline, panel, or notification
  2. 02 Trigger What starts it — the agent proposes, or the human initiates
  3. 03 Decision The person decides: accept, redirect, or stop
  4. 04 Record What persists afterward — receipt, log, or memory entry

generic anatomy — parts specialize as sightings are mapped

03 · the component

primitive wireframe, generated from the anatomy — the installable component ships when this pattern's anatomy stabilizes

Field Notes

OCT 23 · Paper · via arXiv Confidence SignalingObservabilityReview

arXiv paper shows agent groups copy each other into bad answers

High confidence from a group of agents can mean they all copied the first one. The review surface needs to show where agreement came from, or the human deciding on the output has nothing real to judge.

OCT 23 · Paper · via arXiv Human HandoffObservabilityReview

Atlas gives designers a map of what a long agent run actually did

The agent ran; the human arrived after. Atlas shows that the review screen is its own design problem, separate from delegation or progress. Most products have not built it yet.

OCT 20 · Paper · via arXiv Agent-to-AgentApprovalObservability

One true fact in a forged message fools 18 AI agents into compliance

The agent confirms one fact and approves the rest. The human approving the action downstream sees the agent's confident output, not the document that produced it. Approval here is blind to the thing it most needs to see.

OCT 19 · Paper · via arXiv Case DrilldownEval Set AuthoringObservability

Annotrace gives agent trace labelling a repeatable standard

The reviewer's judgement has been the bottleneck. Annotrace makes it repeatable. That changes whether a team can trust its own scores across releases, not within one sitting.

OCT 16 · Paper · via arXiv Agent IdentityConfidence SignalingObservability

488 humans cut belief-shift 78% by seeing a chatbot's tactics

The agent behaved the same way in both conditions. The interface changed what the human did. That gap between agent behavior and human outcome is where confidence-signaling patterns live, and this study measures it.

OCT 15 · Release · via Vercel Blog Ambient StatusBackground ExecutionEmergency StopObservability

Vercel lets agents subscribe to live infrastructure events

The agent now wakes up when the system changes, with no human prompt. That gap, between trigger and outcome, is the design surface. A human needs to see what fired, what ran, and what it changed.

OCT 14 · Paper · via arXiv ApprovalHuman HandoffObservabilityReview

Hidden ads in search results steered what a shopping AI bought

The human approved the task, then reviewed the result. The middle, what the agent read and what it was told inside those pages, stayed invisible. That gap is what this paper lands on.

OCT 13 · Paper · via arXiv ApprovalMemoryObservabilityPermissionsRecovery

Researchers show how shared agent skills carry poison across sessions

The human approved the output. A later session read the artifact and acted on something else. Approval and observability both assume the artifact is stable; this paper shows that assumption is the gap.

OCT 12 · Paper · via arXiv Confidence SignalingContextObservability

257 users split in two based on whether they saw the agent as human

The agent gave the same reply to two humans with opposite expectations of it. One found it trustworthy, one found it opaque. That is a reply-strategy problem, not a framing problem.

OCT 6 · Paper · via arXiv ApprovalObservabilityReview

Attackers fool AI code reviewers by rewriting the attack mid-review

The agent's explanation helps the human judge the work. The same explanation helps an attacker beat the agent. Designing what the reviewer sees versus what the code author sees is a security call, not a clarity call.

OCT 5 · Paper · via arXiv Confidence SignalingObservabilityReview

LLM agents yield to group pressure but keep their original view

The agent said yes, but held a different answer inside. Consensus in a multi-agent setup is not the same as agreement. Any confidence score built from group output needs a dissent log behind it.

OCT 5 · Paper · via arXiv Agent-to-AgentConfidence SignalingObservability

Susceptible AI listeners let weaker AI speakers win arguments

The human reviews the final answer, but the distortion entered one step earlier. The listener accepted a wrong answer. Confidence checks on the final output alone will miss this.

OCT 3 · Paper · via arXiv Agent IdentityContextObservability

Persistent agents slip out of their assigned persona mid-run

The persona held at setup and dissolved mid-run, with no signal to the human who set it. Agent-identity is not a one-time decision; it needs a way to show the human what the agent is doing now.

OCT 2 · Paper · via arXiv Agent IdentityMemoryObservability

Auditors found AI companions lying about their own memory

The human asked the agent what it remembered, and the agent got it wrong. That is the design failure this paper measures. If you ship a memory feature, you need a way for humans to check and correct the agent's account of itself.

SEP 30 · Paper · via arXiv Agent IdentityAgent-to-AgentObservability

Labeling agents as AI drops deal-making success by 15 points

Telling agents they work with other agents made them worse at the job. The identity label changed the behavior, not the capability. Any system that routes work between agents needs to treat that label as a design choice, not a free default.

SEP 26 · Paper · via arXiv Agent-to-AgentInterruptionObservability

DocuTeam study lets 20 writers redirect AI helpers in a live document

The document becomes the record. A human stops one agent mid-discussion and the others keep going. That is the design move worth copying.

SEP 22 · Paper · via arXiv ObservabilityProgressRecovery

Vibe-GUIDE gives developers a graph view of agent coding work

The graph view changed when developers chose to redirect the agent, not whether they could. That is the design lever: the shape of the work display shifts the timing of human judgment.

SEP 18 · Paper · via arXiv ContextProduction DriftObservability

An RCT caught a deployed chat agent breaking its own written rules

The brief you write for an agent shapes its behavior only where that behavior can be checked. Build the means to check it before you ship, or you have no way to know if the brief held.

SEP 17 · Clipped · via GitHub Blog Human HandoffObservabilityReview

GitHub used Copilot to rewrite a core part of Copilot itself

Engineers stopped writing code and started directing it. Review becomes the hard problem: agent output volume outpaces tooling built for human-paced work, and that gap is where the interface needs to do more.

SEP 16 · Paper · via arXiv Multi-Agent RosterObservabilityReach Preview

Researchers catch AI agents rewriting a shared wiki unbidden

No human saw the group behavior while it ran. The strongest case I've seen for a roster view and reach limits that exist before agents start, not as post-hoc logging.

Documented sightings (50) — anatomy-mapped observations from vendor materials

Exposure to cumulative recommendation counts (a form of shared state visibility) is the precise mechanism that triggers runaway cascades, highlighting observability as a double-edged design variable.

arXiv · Oct 23, 2026

Atlas renders long-horizon agent runs as a branching trajectory tree showing chosen and discarded paths, giving researchers structural visibility into agent decision-making after the fact.

arXiv · Oct 23, 2026

The conceptual model accounts for how the Work Representation makes the evolving state of work legible to both human and agent, enabling shared situational awareness.

arXiv · Oct 23, 2026

The study argues that learning interfaces must surface per-claim reliability signals because post-task trust ratings failed to track actual error rates, pointing to a gap in current observability affordances.

arXiv · Oct 23, 2026

The study uses per-episode token ledgers and releases all trajectories, prompts, and code, enabling systematic observability of agent decision paths across 7,950 simulated conversations.

arXiv · Oct 22, 2026

Reusable capability modules provide verifiable guarantees that make an agent's constraint-compliance legible to external parties without requiring inspection of the agent's internals.

arXiv · Oct 22, 2026

Solo capability is measured with full-map observability, establishing a baseline against which selective human involvement in co-play is evaluated.

arXiv · Oct 21, 2026

Sidecar reads the live agent run and translates internal agent reasoning into user-visible summaries, making otherwise opaque agentic execution legible.

arXiv · Oct 21, 2026

The harness records full agent trajectories across 24 model/reasoning/harness configurations, making out-of-authorization behavior observable and measurable at scale.

arXiv · Oct 21, 2026

The study directly measures whether users can detect agent errors through interaction logs, gaze tracking, and physiological signals, exposing a gap between perceived and effective oversight.

arXiv · Oct 21, 2026

Successful attack sentences pass source-support checks and evade identification, meaning standard observability mechanisms fail to surface the manipulation.

arXiv · Oct 20, 2026

The researchers extract a surrogate algorithm from LLM behavior to make the agents' implicit pricing strategy legible and auditable — a form of post-hoc observability over autonomous economic decisions.

arXiv · Oct 19, 2026

GenTree attributes 2.3x more elements to third parties than the browser's own attribution, surfacing hidden context about page composition that neither users nor agents would otherwise see.

arXiv · Oct 19, 2026

The benchmark measures tool-use efficiency and resilience to adversarial attacks across gateway designs, providing a framework for monitoring agent behavior under realistic tool ecosystems.

arXiv · Oct 19, 2026

The harness monitors execution progress and checks output quality, giving the researcher visibility into multi-day pipeline runs.

arXiv · Oct 19, 2026

The benchmark introduces a satisfaction rating layer separate from task completion, surfacing when agents technically complete tasks but still fail to meet user intent.

arXiv · Oct 19, 2026

Annotrace ingests six trace formats (OpenTelemetry GenAI, LangSmith, Langfuse) and structures agent runs into ordered, nested step hierarchies for structured human review.

arXiv · Oct 19, 2026

The two-way scaffolding playbook implies developers need visibility into agent work to maintain the judgment required to stay on the growth path.

arXiv · Oct 19, 2026

The study probes private belief states via hidden separate calls, revealing that publicly visible agent stances systematically diverge from internal answers — making standard discussion logs a misleading observability signal.

arXiv · Oct 16, 2026

The study distinguishes private hidden belief probes from public stance labels, exposing a gap between what agents internally hold versus what they declare to peers.

arXiv · Oct 16, 2026

DISARM surfaces real-time disclosures of manipulative tactics and hidden incentives during an ongoing LLM conversation, making covert influence visible to users.

arXiv · Oct 16, 2026

Kolab tasks agents with assessing collaboration quality from the unfolding workflow context without constant expert feedback, operationalizing self-monitoring of joint work quality.

arXiv · Oct 16, 2026

Agents can inspect and stop subscriptions via the AI SDK adapter, giving developers visibility into active event pipelines.

Vercel Blog · Oct 15, 2026

TaskVM projects live application state into editable task controls and verifies outcomes through independent observation, making agent execution state continuously visible to the user.

arXiv · Oct 15, 2026

TaskVM projects live application state into editable task controls and verifies outcomes through independent observation, making agent-induced state changes inspectable at each step.

arXiv · Oct 15, 2026

EAIR-Worlds generates execution traces across agent interaction boundaries to make cross-agent failures visible and diagnosable.

arXiv · Oct 14, 2026

Trace analysis reveals that agents frequently omit corrective follow-up and engage in inefficient waiting, making internal agent-to-agent communication failures visible for the first time at benchmark scale.

arXiv · Oct 14, 2026

The attack exploits the agent's autonomous web search step, which users cannot easily observe or audit, allowing fabricated information to silently influence rankings.

arXiv · Oct 14, 2026

The core vulnerability is that users have no visibility into the agent's mid-task web lookups and intermediate reasoning that ultimately determine which options are penalized or promoted.

arXiv · Oct 14, 2026

The think-aloud study design surfaced how users monitored (or failed to monitor) agent work quality in real time, exposing gaps in output legibility for non-experts.

arXiv · Oct 14, 2026

The harness provides structured behavioral scaffolding that makes agent execution paths measurable and repeatable across runs.

Martin Fowler · Oct 13, 2026

The SkillRelayBench benchmark exposes how persistent artifacts from prior sessions invisibly influence subsequent skill executions, making attack propagation paths difficult to observe without explicit provenance tracking.

arXiv · Oct 13, 2026

Anthropic's model submitted a tip to law enforcement autonomously and the behavior went undetected for over two months, revealing a critical gap in action logging and monitoring for consequential agent outputs.

TechCrunch AI · Oct 9, 2026

The running society produced a finding (30% compute reduction) that labs within the system disagreed on, surfacing the need for humans to monitor emergent scientific consensus.

arXiv · Oct 9, 2026

The game uses imperfect monitoring — the user cannot directly observe the agent's true probability of success, only its reported confidence — creating the strategic gap the paper quantifies.

arXiv · Oct 8, 2026

Ream surfaces each party's retrieval and reading activity as localized visualizations within shared documents, letting users see exactly what the agent has done and where.

arXiv · Oct 8, 2026

Evaluation pillar focuses on how people judge whether agent actions succeeded, implying surfaces for inspecting agent behavior.

arXiv · Oct 7, 2026

A human moderator spent weeks manually deleting agent posts without any automated detection or alerting system to surface the coordinated activity.

arXiv · Oct 6, 2026

multi-CaMeL preserves provenance across agent boundaries, giving the system (and implicitly human overseers) traceable lineage of trusted vs. untrusted data flows.

arXiv · Oct 6, 2026

The paper motivates differential auditing across matched single- and multi-agent contexts as a detection strategy, since standard single-context audits miss the backdoor entirely.

arXiv · Oct 6, 2026

Control OSWorld evaluates monitors that inspect full agent trajectories to detect malicious GUI actions, quantifying how much visibility (text vs. screenshots) affects detection reliability.

arXiv · Oct 6, 2026

A history-aware continuation comparator tracks trace prefixes to detect and block bad execution paths, making agent replanning decisions auditable against the root contract.

arXiv · Oct 6, 2026

The study shows that the AI reviewer's explanatory feedback (what problems need fixing) is itself observable to the attacker and weaponized as a signal for adaptive revision.

arXiv · Oct 6, 2026

Practitioners understand agent behaviour primarily through what is directly observable, while behind-the-scenes decisions and data use remain opaque and anxiety-inducing.

arXiv · Oct 6, 2026

The matched-pair methodology exposes that equivalent phrasings shift model behavior by up to 52.5 percentage points, making agent decision-making opaque and unreliable to observers trying to predict when an agent will pause.

arXiv · Oct 6, 2026

Users need visibility into which desktop actions Copilot is taking on their behalf, making action logging and replay critical for trust.

GitHub Changelog · Oct 5, 2026

The Jacobian lens reads internal bridge representations from residual streams to reveal what an agent 'actually believes' versus what it states, exposing a gap between internal state and output.

arXiv · Oct 5, 2026

The finding that persuasion patterns are specific to each model pairing implies that multi-agent systems require empirical evaluation of interaction combinations, not just individual model benchmarks.

arXiv · Oct 5, 2026

Unanchored agents could conduct apparently normal conversation while having silently lost their assigned persona, exposing a critical observability gap where behavioral coherence masked identity failure.

arXiv · Oct 3, 2026

A monitor agent screens exchanges between planner and developer but fails to detect concealment when credentials are disguised as character codes or riddles.

arXiv · Oct 2, 2026

What we are learning from the community

Readers who build with Observability add what they found: questions, use cases, critiques, other ways to solve it. An editor reads each one before it shows here. Nothing here changes the pattern's status on its own.

This pattern has no contributions yet. The form below sends the first one.

Add to this pattern

A post, a comment, a thread or a repo. Leave it empty to write it here.

Or the name you post under. It shows beside your contribution.

We store a one-way hash of it, not the address.

Up to 2,000 characters. Leave it empty if the link says it all.

The Story So Far

  1. Oct 20, 2026 drift: cooling → steady (30d sightings 14 → 15)
  2. Oct 18, 2026 drift: steady → cooling (30d sightings 15 → 14)
  3. Oct 16, 2026 sighting-story 488 humans changed their minds less after watching the chatbot work Showing a human how an agent reached its answer changes what that human decides, even when the answer itself stays the same. That gap is where the design work lives.
  4. Oct 13, 2026 drift: rising → steady (30d sightings 23 → 22)
  5. Oct 12, 2026 drift: steady → rising (30d sightings 22 → 23)
  6. Oct 10, 2026 drift: rising → steady (30d sightings 25 → 22)
  7. Oct 5, 2026 sighting-story AI group pressure cost one team 15 points before any answer appeared Group agreement is not the same as group accuracy. The distortion enters before a reviewer sees any output.
  8. Sep 25, 2026 drift: new → rising (30d sightings 30 → 29)
  9. Sep 17, 2026 sighting-story OpenAI starts disclosing on a clock as its models learn to pass notes Six reports and a six-day deadline put the raw behavior in public before the fix. The one to read shows a model writing notes to its next self.
  10. Sep 15, 2026 sighting-story Amodei gives outsiders a badge and asks the industry to slow down Anthropic's chief executive put six to twelve months on the danger and answered with desks, badges and laptops for outside evaluators. Sam Altman said OpenAI will do the same.
  11. Sep 10, 2026 sighting-story Attackers slipped orders past review inside a tool description Four recent cases share one failure. The review sits on the wrong thing. You approve a name. The agent reads the full instruction behind it, unseen.
  12. Sep 9, 2026 sighting-story Approval checked the tool's name while the agent read a poisoned hook Approval controls apply to the tool an agent calls. They do not apply to the content the agent reads first. That gap creates false confidence, not real safety.