Why alerting on AI cost is harder than it sounds

The pattern is consistent across stories I've seen: $80 surprise bills, $607 Replit runaways, $7,000-in-a-day Cursor incidents, and a recently-reported $500M Claude bill from a single enterprise. Every one of them ended with the same line — we found out from the invoice.

The problem isn't that alerting tools don't exist. It's that AI cost lives in too many places. Your team is probably running:

• Cloud-hosted coding tools — Cursor, Claude Code, Codex CLI on developer machines
• Direct API calls — your application code calling Anthropic, OpenAI, Gemini directly
• Production AI features — RAG pipelines, support agents, summarization
• Background automation — scheduled scripts, cron jobs, batch processing

A spend alert at the Anthropic dashboard catches some of this. A Datadog alert catches some of it. Neither catches all of it. And the ones that fall through the gaps tend to be the most expensive.

Let me walk through each layer.

Layer 1 — Vendor-level alerts

This is the bare minimum. Every AI vendor offers some form of usage cap and alert. Set them today if you haven't.

Anthropic Console

Go to your Anthropic workspace settings → Limits. You can set:

• Monthly spending limit per API key — hard cap that stops requests when exceeded
• Email alerts at 50%, 75%, 90% of your limit
• Per-workspace limits for multi-team setups

What this catches: an entire workspace overrunning its budget over the course of a month.

What this misses: a single agent running 30x more than expected for two days. The monthly limit is too coarse — a $5K monthly cap doesn't fire if a runaway agent burns $300 in a weekend.

OpenAI Platform

Settings → Billing → Usage limits. Two settings:

• Hard limit — requests stop when reached
• Soft limit — email alert, requests continue

Set the soft limit at ~50% of your expected monthly spend. Set the hard limit at 150%.

Same limitation as Anthropic: monthly granularity. A runaway script during off-hours will hit the soft limit, but only after meaningful spend has already occurred.

Cursor

For team accounts: Admin settings → Billing → Set monthly budget. Per-user spending caps are configurable but require Enterprise pricing.

Cursor changed their pricing model in mid-2025, which created surprise bills for many users. The lesson: if your vendor changes pricing models, your alert thresholds become stale overnight. Re-verify monthly.

What vendor-level alerts actually catch

Vendor alerts are good at one thing: stopping catastrophic monthly overruns. They are bad at almost everything else:

• They don't see usage at the per-task or per-user level (only per-key)
• They don't catch fast spikes within their reporting cadence
• They don't correlate across vendors (Anthropic alerts know nothing about your Cursor spend)
• They don't surface why spend is climbing, only that it is

This is your seatbelt. Necessary, not sufficient.

Layer 2 — Proxy-level alerts (for production AI)

If your application makes API calls to LLMs in production, you need a proxy or instrumentation layer that sees each call.

Option A — Helicone

Route your API calls through Helicone's proxy by changing the base URL:

from openai import OpenAI
client = OpenAI(
    api_key="sk-...",
    base_url="https://oai.helicone.ai/v1",
    default_headers={
        "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
    }
)

Once requests flow through Helicone, you can set alerts on:

• Cost per hour above a threshold
• Request rate spike vs baseline
• Error rate above 1%
• Specific cost-per-user thresholds

Helicone alerts go to Slack, email, or webhook.

Option B — LiteLLM (self-hosted)

If you don't want a third-party proxy, run LiteLLM as a gateway. Same pattern: your code calls LiteLLM, LiteLLM calls the upstream provider, you get telemetry.

# config.yaml
model_list:
  - model_name: claude-3.5-sonnet
    litellm_params:
      model: anthropic/claude-3-5-sonnet-20241022
      api_key: os.environ/ANTHROPIC_API_KEY
general_settings:
  alerting:
    - type: slack
      webhook_url: https://hooks.slack.com/...
      alert_types: ["spend_threshold", "rate_limit"]

LiteLLM publishes Prometheus metrics out of the box. If you're already on Grafana, this is the cleanest path.

Option C — Custom instrumentation

If you want full control, wrap your LLM client calls with your own metrics. Pseudocode:

async def call_llm(prompt, user_id, feature):
    start = time.time()
    response = await openai_client.chat.completions.create(...)

    metrics.emit({
        "tokens_in": response.usage.prompt_tokens,
        "tokens_out": response.usage.completion_tokens,
        "cost": calculate_cost(response.model, response.usage),
        "user_id": user_id,
        "feature": feature,
        "duration_ms": (time.time() - start) * 1000,
    })

    return response

Push the metrics into Prometheus, Datadog, or whatever observability stack you use. Then alert on the metric you care about.

What proxy-level alerts actually catch

This layer is where you catch production AI cost anomalies — a customer triggering Opus 100x in an hour, a feature suddenly costing 10x normal, a model routing bug. It's also where you get per-feature and per-customer attribution that vendor consoles don't surface.

What proxy-level alerts miss

Everything that doesn't flow through your proxy. That's a much bigger category than most teams realize:

• Claude Code sessions on developer laptops (never touch your proxy)
• Cursor sessions on developer laptops (never touch your proxy)
• Engineers debugging in the Anthropic Console directly
• Cron jobs and scheduled scripts that someone forgot to instrument
• The pipeline you built last month for a one-off task and forgot about

If your team's AI usage is mostly developer-laptop coding tools, proxy alerts won't save you. They're the wrong layer for that surface.

Layer 3 — Developer-machine alerts (the missing layer)

This is the layer most teams don't have, and where the worst incidents happen. Why? Because coding tools like Claude Code, Cursor, and Codex CLI run locally on developer laptops — their API calls never flow through your application's proxy or instrumentation.

What's available today

For Claude Code specifically:

ccusage (free) — Parses Claude Code's local JSONL session files. Run ccusage --watch and pipe to a notification:

ccusage --watch --threshold 50 | while read line; do
  if [[ $line == *"exceeded"* ]]; then
    curl -X POST $SLACK_WEBHOOK -d "{\"text\":\"$line\"}"
  fi
done

Works for one developer at a time. No team rollup. You'd need this script on every machine.

Claude Code Usage Monitor — Open-source CLI dashboard similar to ccusage. Same single-developer limitation.

For Cursor: no equivalent OSS tool that I'm aware of. Cursor's admin API gives you team-level usage data on Enterprise plans, but it's not real-time and doesn't support custom alerting.

For Codex CLI: similar gap. Limited public tooling for usage monitoring.

The DIY pattern

If you're willing to build it yourself, the architecture looks like this:

• Collector runs on each developer machine, watching the local session directory
• Push aggregated data to a central endpoint every minute
• Server aggregates across the team, runs anomaly detection
• Alert when individual or team usage exceeds thresholds

Realistic build time: 3-5 days for a v1, then ongoing maintenance every time a tool vendor changes their session format.

What this layer catches that nothing else does

Three classes of incident only show up at the developer-machine layer:

• Runaway local pipelines. An agent or script left running on a developer's machine.
• Misuse patterns. A developer using Opus when Sonnet would do, or running 4x the team average without realizing it.
• The "I forgot about that script" pattern. Background processes that someone wrote for a one-off task and never cleaned up.

These all show up on the developer's machine, never on your proxy, and only as aggregate spend on the vendor console.

Putting it together — what an actually-comprehensive setup looks like

Most teams have layer 1 alone. Some teams have layers 1 and 2. Very few have all three.

The actual coverage map:

If your AI usage is mostly server-side production calls, layers 1 + 2 cover you. If your team has multiple coding agent tools running on laptops, you need all three. If you're still weighing the specific tools, we compared them in how to track Claude Code costs across a team.

The honest reality: layer 3 is where the gap is. Most teams haven't built it because it's the hardest of the three — single-developer tools don't aggregate, custom builds take a week, and no major observability vendor covers it yet.

This is the gap AIWatcher fills. We're the operational layer that handles layer 3 for you across Claude Code, Cursor, Codex CLI, Cline, and Windsurf — with real-time anomaly alerts, no instrumentation, and install in under an hour. We also handle layer 2 via our SDK if you want both layers in one dashboard. AIWatcher is in design partner mode in 2026 with mid-market AI-native SaaS companies. Request access →

Closing

If you take one action from this post: go set vendor-level limits this afternoon. They're the easiest win and most teams haven't done it yet.

If you take a second: pick the layer where your biggest gap is. If your team runs coding agent tools on laptops and your only alert is the vendor monthly cap, you're one runaway script away from finding out the wrong way.

The teams that handle this well aren't doing anything magical. They've just acknowledged that AI cost lives in more than one place, and they've put alerts at every layer.

Danny is the founder of AIWatcher, the operational layer for AI agents. We build the developer-machine layer most teams haven't gotten to yet.