AI Agents with Tool Calling: Architecture Patterns That Actually Work

The first agent I built for a client was a customer-support triage bot that could read tickets, search the knowledge base and draft replies. In testing it worked beautifully. On its second day in production it entered a loop between two tools, made 340 API calls in nine minutes and cost more than the previous month's total bill. The fix was not a better prompt. It was architecture. This post is that architecture.

Strip away the hype: what an agent is

An LLM agent is a program that repeatedly (1) sends the conversation so far plus a list of available tools to a model, (2) receives either a final answer or a request to call one of those tools with specific arguments, (3) executes the call in your code, and (4) appends the result to the conversation. That loop is the entire secret. The model never executes anything; your runtime does, which means your runtime is where safety, cost control and observability live.

Pattern 1: The ReAct loop (default choice)

"Reason + Act": the model thinks briefly, picks a tool, observes the result, thinks again. It handles open-ended tasks where you cannot predict the sequence of steps. Here is a production-shaped loop in TypeScript using the Anthropic SDK. The shape is identical for OpenAI's function calling; only the field names change.

// agent/loop.ts
import Anthropic from '@anthropic-ai/sdk';
import { tools, execute } from './tools';        // schemas + implementations
import { StepLog } from './step-log';

const client = new Anthropic();

export interface Budget { maxSteps: number; maxTokens: number; maxMs: number; }

export async function runAgent(task: string, budget: Budget, log: StepLog) {
  const messages: Anthropic.MessageParam[] = [{ role: 'user', content: task }];
  const started = Date.now();
  let tokens = 0;

  for (let step = 0; step < budget.maxSteps; step++) {
    if (Date.now() - started > budget.maxMs) return fail('time budget exceeded', log);
    if (tokens > budget.maxTokens)           return fail('token budget exceeded', log);

    const res = await client.messages.create({
      model: 'claude-sonnet-4-5',
      max_tokens: 1024,
      system: SYSTEM_PROMPT,
      tools,
      messages,
    });
    tokens += res.usage.input_tokens + res.usage.output_tokens;

    const toolUses = res.content.filter(b => b.type === 'tool_use');
    messages.push({ role: 'assistant', content: res.content });

    if (res.stop_reason !== 'tool_use' || toolUses.length === 0) {
      const text = res.content.filter(b => b.type === 'text').map(b => b.text).join('');
      await log.finish({ answer: text, tokens, steps: step + 1 });
      return { ok: true, answer: text };
    }

    // Execute every requested tool (they are independent, so run in parallel).
    const results = await Promise.all(toolUses.map(async (call) => {
      const cached = await log.getResult(call.id);                 // idempotency on resume
      const result = cached ?? await guardedExecute(call.name, call.input, log);
      await log.record({ step, call, result });
      return { type: 'tool_result' as const, tool_use_id: call.id, content: JSON.stringify(result) };
    }));

    messages.push({ role: 'user', content: results });
  }
  return fail('step budget exceeded', log);
}

async function guardedExecute(name: string, input: unknown, log: StepLog) {
  const t = tools.find(t => t.name === name);
  if (!t) return { error: `unknown tool ${name}` };
  const parsed = t.schema.safeParse(input);                       // Zod validation
  if (!parsed.success) return { error: 'invalid arguments', issues: parsed.error.issues };
  if (t.requiresApproval) {
    const approved = await log.requestApproval(name, parsed.data);   // pauses the run
    if (!approved) return { error: 'action rejected by human reviewer' };
  }
  try { return await execute(name, parsed.data); }
  catch (e) { return { error: (e as Error).message }; }          // errors go BACK to the model
}

async function fail(reason: string, log: StepLog) {
  await log.finish({ error: reason });
  return { ok: false, error: reason };
}

Three deliberate choices in that loop:

• Errors are returned, not thrown. A tool failure becomes a tool result the model can read and route around. Throwing kills a run that might have recovered by trying a different query.
• Invalid arguments are explained. Sending Zod's issue list back produces a corrected call on the next step almost every time.
• Results are persisted by tool_use_id. If the process dies at step 7 of 12, the resumed run replays steps 1 to 6 from the log at zero cost. This is the same idempotency discipline I use for queue consumers.

Defining tools the model can actually use

// agent/tools.ts
import { z } from 'zod';
import { zodToJsonSchema } from 'zod-to-json-schema';

const defs = [
  {
    name: 'search_tickets',
    description: 'Search support tickets by free text. Returns up to 10 tickets with id, subject, status and a 200-char excerpt.',
    schema: z.object({ query: z.string().min(2).max(120), status: z.enum(['open','pending','closed','any']).default('open') }),
    requiresApproval: false,
  },
  {
    name: 'search_docs',
    description: 'Semantic search over the help centre. Returns passages with source URLs. Use before drafting any answer.',
    schema: z.object({ question: z.string().min(5).max(300) }),
    requiresApproval: false,
  },
  {
    name: 'send_reply',
    description: 'Send a reply on a ticket. IRREVERSIBLE. Only call after search_docs confirmed the answer.',
    schema: z.object({ ticketId: z.number().int().positive(), body: z.string().min(20).max(2000) }),
    requiresApproval: true,
  },
] as const;

export const tools = defs.map(d => ({
  name: d.name,
  description: d.description,
  input_schema: zodToJsonSchema(d.schema) as any,
  schema: d.schema,
  requiresApproval: d.requiresApproval,
}));

The search_docs tool is the RAG hybrid search wrapped in a function. If you want these tools usable from Claude Desktop or Cursor as well as your own runtime, expose the same definitions through an MCP server; the schemas are identical.

Pattern 2: Planner-executor for long jobs

When a task has twenty steps (migrate 300 product descriptions, audit every page of a site), a single ReAct loop drifts: the context fills with old tool results and the model forgets the goal. Split it. A planner call produces an explicit, structured list of subtasks; an executor runs each subtask as its own short ReAct loop with a fresh context, and a small reducer merges results.

const Plan = z.object({
  goal: z.string(),
  steps: z.array(z.object({ id: z.string(), instruction: z.string(), dependsOn: z.array(z.string()).default([]) })).max(30),
});

export async function runPlanned(task: string) {
  const plan = await structuredCall(Plan, `Break this task into independent steps. Task: ${task}`);
  const results = new Map<string, unknown>();

  for (const batch of topologicalBatches(plan.steps)) {                  // parallel where deps allow
    await Promise.all(batch.map(async (s) => {
      const context = s.dependsOn.map(d => `Result of ${d}: ${JSON.stringify(results.get(d))}`).join('\n');
      const r = await runAgent(`${s.instruction}\n\n${context}`, { maxSteps: 8, maxTokens: 40_000, maxMs: 90_000 }, new StepLog(s.id));
      results.set(s.id, r);
    }));
  }
  return structuredCall(FinalReport, `Summarise these results for the user:\n${JSON.stringify([...results])}`);
}

Each executor gets a tight budget. A step that goes wrong fails alone and can be retried alone. On the product-migration job this cut token spend by roughly 60% versus one long loop, mostly because each step's context stayed small.

Pattern 3: Router for classify-then-dispatch

Sometimes you do not want an agent at all. If the input falls into one of five known categories, do one cheap structured call to classify it, then run deterministic code or a specialised prompt for that category. Support triage, lead qualification and content moderation all fit here. A router is faster, cheaper and far easier to test than a loop, and it is my first suggestion whenever a client says "we need an agent".

const Route = z.object({ intent: z.enum(['billing','bug','feature','spam','other']), confidence: z.number().min(0).max(1) });

export async function handleTicket(ticket: Ticket) {
  const { intent, confidence } = await structuredCall(Route, `Classify this ticket:\n${ticket.body}`, { model: 'claude-haiku-4-5' });
  if (confidence < 0.7) return escalateToHuman(ticket, 'low confidence');
  switch (intent) {
    case 'billing': return billingWorkflow(ticket);            // deterministic code
    case 'bug':     return runAgent(`Investigate and draft a reply for bug report #${ticket.id}`, BUG_BUDGET, new StepLog(ticket.id));
    case 'spam':    return ticket.close('spam');
    default:        return escalateToHuman(ticket, intent);
  }
}

Guardrails that matter in production

Choosing a pattern

• Known categories, predictable handling → Router. Start here.
• Unknown sequence, under ~10 tool calls → ReAct loop with tight budgets.
• Large, decomposable job → Planner-executor with per-step loops.
• Real-time and latency-critical → probably not an agent; a single structured call and deterministic code.

The support bot that melted its budget now runs as a router in front of a bounded ReAct loop, with send_reply behind human approval. It handles about 70% of tickets end-to-end, costs a fraction of the original design, and has not looped once since. If you are planning something similar and want a second pair of eyes on the architecture, that is exactly the kind of work I take on.