The control loop for AI work: what it is, and why observability isn't it

"Usage resets at 7PM."

It was 2 in the afternoon, and half my task list was still sitting there. Where did the usage go? One lazy prompt. I'd typed "why isn't this working?" under a stack trace, the agent went digging through the whole codebase looking for suspects, and because every file an agent opens stays in its context, and every turn resends that entire context, one prompt turned into 150,000 tokens. The task succeeded. It just cost about 30x what it needed to. The waste never showed up as a bill. It showed up as an afternoon I didn't get back.

Here's the version that happens inside your product. A feature starts failing validation one night, malformed JSON on certain customer documents, and the on-call mitigation is the classic one: bump the model. Fable powers through the cases Sonnet was tripping on, the incident closes, everyone moves on. The override was supposed to be temporary. Three weeks later that path is still budgeted at Sonnet prices and billed at Fable prices, and the feature looks healthier than ever: error rate down, latency fine, every dashboard green. Add the quieter versions of the same shape, calls succeeding on the third retry at triple the cost, a workflow taking forty steps for certain customers instead of five, and the bill jumps 40% while nobody can say which feature or which agent did it.

Nothing in either story broke. That's the problem. Every monitoring stack is built to watch for failure. Success is the one condition no monitoring investigates.

A control loop is the thing built to investigate it.

Why a loop, and not another dashboard

The case for a loop is the gap in the middle of what you already have.

Your dashboards (ccusage, vendor consoles, custom Datadog builds) tell you what happened after the money is spent. They investigate failures, and they have nothing to say about sessions that succeeded expensively.

Your gateway can block and route API traffic, on the one path routed through it. It never sees the coding agents on your developers' laptops.

And developer vigilance runs out. Anthropic's own telemetry showed developers approving roughly 93% of Claude Code's permission prompts, which is why they shipped Auto Mode, a classifier that auto-approves safe actions. Asking a human to review every action doesn't produce review. It produces clicking.

So: visibility after the fact, enforcement on one path, attention that runs out. What's missing wraps the whole lifecycle of the work. That's a control loop. Five steps:

• PLAN — score the work before it runs
• WATCH — track what you can't see from inside the session
• CONTROL — gate the moves that matter
• PROVE — tie every session to what it produced
• IMPROVE — learn from all of it, so the next session starts smarter

PLAN

Every prompt gets scored for scope and risk before the first token. The lazy "why isn't this working?" gets flagged and rewritten into a scoped brief: these files, this goal, stop when the test passes. AIWatcher's preflight does this across Claude Code, Codex, and Cursor, and at the workflow level in production. It's the only intervention in the stack that costs zero tokens.

WATCH

The loop tracks what's invisible from inside a session: context fill, junk from dead ends, usage left against the work remaining. When a session bloats, AIWatcher offers a handoff brief so the next run starts clean, and it works across tools, Claude Code to Codex or Cursor, where native resume can't follow. In production, it surfaces silent retries as the gap between cost per request and cost per success.

CONTROL

Bulk deletes, live-data migrations, force pushes. Looping sessions. Runaway workflows. AIWatcher gates these with Allow, Brief, or Cancel, one policy across Claude Code, claude.ai, Codex, Cursor, and Claude Desktop. The bar: a gate you see a few times a month and agree with, not a prompt you rubber-stamp ten times a day.

PROVE

Every session gets tied to what it produced. On the laptop: a local ledger linked to git, privacy-safe evidence from commit and test signals, no prompt text stored. Cost per surviving change, not cost per token. In production: every call tagged by feature, customer, and workflow, so a 40% bill jump decomposes in a filter, not a week of grepping logs. The model nobody switched back is the first row.

IMPROVE

Every DIY dashboard in this space is hand-tuned forever; the practitioners building the best ones say so themselves. The loop tunes itself: preflight adapts per repo and per developer as it learns which briefs saved tokens, and every gate that fired, right or not, sharpens what fires next time. A control system that doesn't learn is a rulebook. Rulebooks go stale.

What a control loop is not

It's not a replacement for your observability: keep Datadog, Helicone, Langfuse. The loop replaces none of your observability and all of your blind spots. It's not a spend cap; hard dollar thresholds are gateway territory; the loop works alongside them. And it's not anomaly detection, because the loop's whole premise is that the expensive thing usually is normal. That's why nothing flags it.

The short version

One loop, two surfaces: developer laptops (Claude Code, Codex, Cursor, read locally, nothing leaves the machine) and inside your product (one line per LLM call, metadata only, never your prompts). Observability tells you what happened. A gateway stops one thing on one path. A control loop runs the whole cycle, on both surfaces, and learns from every control it applies.

Developers: AIWatcher OSS is free, open source, and local-first. See what your sessions have been costing you.

If you run a team or ship AI features: getaiwatcher.com. The loop is the product.