The Agentic System That Worked Perfectly — Until It Didn't

“The agent wasn’t malfunctioning. It was doing exactly what we told it to do. That was the problem.”

The Setup

In my current role I architected an agentic AI pipeline to handle routine cloud infrastructure provisioning requests. The goal: reduce the load on the platform engineering team for repetitive IaC scaffold requests — new resource groups, naming-convention-compliant storage accounts, standard tagging policies.

The system used four agents in sequence:

1. Intent Parser — extracted structured intent from free-text Jira tickets
2. IaC Generator — produced Terraform HCL matching internal naming conventions
3. Policy Gate — ran Checkov validation, blocked non-compliant output
4. Executor — ran terraform apply in a sandboxed Azure subscription

Three weeks of production operation. 89 requests processed. Zero incidents. The team loved it.

The Mess

On a Tuesday morning, the system processed 47 consecutive provisioning requests in 6 minutes.

The Executor created 47 resource groups. Naming convention: correct. Tags: correct. Terraform state: valid. Policy Gate: all passed.

The 47 requests were not legitimate. They were synthetic. A developer testing an upstream Jira webhook integration had sent test payloads to the production endpoint. The Intent Parser correctly parsed them as provisioning requests. The IaC Generator dutifully produced valid HCL for each. The Policy Gate passed them all. The Executor executed.

The system worked perfectly. That was the incident.

# Jira webhook test payload that triggered the cascade
POST /api/agent/provision
Content-Type: application/json
{
  "issue_key": "TEST-001",
  "summary": "Provision new environment for load testing",
  "description": "Create a standard resource group with storage and key vault",
  "reporter": "test-automation-bot@internal",
  "labels": ["infrastructure", "provisioning"]
}

The Intent Parser did not distinguish between test-automation-bot and a human engineer. It was not designed to. The word “test” in the summary was not a signal — plenty of legitimate requests include test environments.

47 iterations of this payload, each with a sequential ticket number. 47 resource groups, each created in 4.2 seconds. Total elapsed time: 6 minutes 14 seconds. Total cleanup time: 4.5 hours.

---

What the Agentic Architecture Was Missing

The post-mortem identified three systemic gaps:

Gap 1: No Request Origin Verification

The agent accepted requests from any authenticated webhook endpoint. We had authentication — the webhook required a valid HMAC signature. We did not have authorisation — verification that the request origin was a human-raised ticket versus a test harness.

Gap 2: No Volume Rate Limiter on the Executor

The Policy Gate ran per-request. There was no agent-level rate limiter that would have flagged: 47 provisioning requests from the same reporter in 6 minutes is statistically anomalous.

This is the circuit breaker failure. I had implemented circuit breakers for LLM inference (trip on 5 consecutive 429 errors). I had not implemented circuit breakers for downstream side-effects — the write operations on Azure.

Gap 3: No Dry-Run Staging Gate

Every execution went directly to terraform apply. There was no staging step where a human reviewed the Executor’s intent before it ran. For repetitive requests, this was acceptable. For first-time requestors or novel patterns, it was not.

---

The Fix: Stateful Circuit Breaker + Velocity Limiter

// src/agents/executor/circuit-breaker.ts

interface ExecutorState {
  provisionsLast5Min: number;
  lastProvisionTimestamp: number;
  fsm: 'CLOSED' | 'TRIPPED' | 'HALF_OPEN';
  tripReason?: string;
}

async function checkExecutorCircuit(
  kv: KVNamespace,
  requesterId: string,
): Promise<{ allowed: boolean; reason?: string }> {
  const key = `executor:rate:${requesterId}`;
  const raw = await kv.get(key);
  const state: ExecutorState = raw
    ? JSON.parse(raw)
    : {
        provisionsLast5Min: 0,
        lastProvisionTimestamp: 0,
        fsm: 'CLOSED',
      };

  const now = Date.now();
  const windowMs = 5 * 60 * 1000; // 5 minutes

  // Reset window if expired
  if (now - state.lastProvisionTimestamp > windowMs) {
    state.provisionsLast5Min = 0;
  }

  // HARD LIMIT: 5 provisions per requester per 5 minutes
  if (state.provisionsLast5Min >= 5) {
    state.fsm = 'TRIPPED';
    state.tripReason = `Rate limit: ${state.provisionsLast5Min} provisions in 5 min window`;
    await kv.put(key, JSON.stringify(state), { expirationTtl: 3600 });
    return { allowed: false, reason: state.tripReason };
  }

  // ANOMALY CHECK: first-time requester with >1 request in flight
  if (state.provisionsLast5Min === 0) {
    // Allow first request, flag the second within 60s as suspicious
  }

  state.provisionsLast5Min += 1;
  state.lastProvisionTimestamp = now;
  await kv.put(key, JSON.stringify(state), { expirationTtl: 3600 });
  return { allowed: true };
}

Additionally: Human-in-the-Loop Staging Gate

For any requester whose reporter field matches non-human patterns (bot accounts, test automation users), all generated Terraform plans now go to a staging queue requiring manual approval before execution.

// Non-human requester detection
const isHumanRequester = (reporter: string): boolean => {
  const botPatterns = [/-bot@/, /automation/, /test-/, /ci-/, /jenkins/];
  return !botPatterns.some((pattern) => pattern.test(reporter.toLowerCase()));
};

The Architecture Principle This Violated

Every agentic system tutorial shows how to chain agents. None of them adequately cover blast radius containment — what happens when the agents work correctly but the inputs are wrong.

A circuit breaker for LLM inference protects against API rate limits. That is protecting the agent. What I had missed was protecting the environment the agent operates in — rate-limiting the downstream side-effects, not just the LLM calls.

The distinction matters. The LLM was not the problem. The automation framework trusted every correctly-formatted request without questioning the volume or pattern.

Key Takeaway

Before deploying any agentic system that writes to production infrastructure, answer this question explicitly: What is the maximum blast radius of a malformed input batch? Then build a circuit breaker sized for that blast radius — not for LLM rate limits.

The human-in-the-loop gate is not a concession to AI risk anxiety. It is an engineering control for ambiguous input space. Automate confidently within the envelope you’ve validated. Require human approval at the edges of that envelope.

---

Architecture and decisions: mine. Debugging sessions at odd hours: mine. AI assistance: structure, syntax, first draft. — Sachin