AI Agent Index
Home/Guides/How to Build an AI Agent
GuideEngineeringUpdated November 2026 by Heather MacAveliaIndependently Reviewed

How to Build an AI Agent That Works in Production (2026)

Many AI agents people are building right now are not agents. They are workflows pretending to be agents, and the confusion costs teams weeks of unnecessary engineering. Anthropic's December 2024 research note Building Effective Agents draws the line clearly: a workflow is a predefined LLM-and-tool chain where each step is decided in advance. An agent is a system where the LLM dynamically decides what to do next until the task is done. Workflows are simpler, cheaper, more reliable, and easier to debug. For many tasks they are the right answer. Anthropic's advice is to find the simplest solution possible and add complexity only when it is needed.

This guide covers what you actually need to build a production AI agent: the four components every agent has, real TypeScript code for the tools and the reasoning loop, the build vs buy decision framework that many teams skip, honest cost math at production scale, and five failure modes to plan for upfront.

Everything in this guide assumes you have already determined you need a real agent, one where the LLM must dynamically decide between multiple tools based on what it discovers at runtime. If you are not certain that is what you need, read the workflow vs agent section first. Many teams who think they need an agent actually need a well-structured workflow with one or two LLM calls, which is dramatically easier to build, maintain, and debug.

The guide uses a customer support triage agent as the running example throughout. It is concrete enough to illustrate real decisions but general enough that the patterns apply across many agent use cases. All code is TypeScript using the Anthropic SDK. Other providers' APIs use a similar structure.

Before you read further: the brutal build-vs-buy decision is the next section. Many teams who think they need to build an agent should buy one instead. Search the Index first.

Build or buy: be honest with yourself

Engineering time is expensive, so run the numbers before you build. A worked example with illustrative figures: assume a senior developer costs $200K a year fully loaded, about $4,000 per week, and that a simple agent takes 2-4 weeks to reach a first version and another 4-8 weeks to harden for production. That is $24K-48K of engineering cost before maintenance, monitoring, and iteration. Against that, a hypothetical off-the-shelf tool costing $200 a month that solves 80% of your use case would take a decade of subscriptions to cost as much. That comparison changes with volume. A support agent priced per resolution, for example, can cost more at this guide's 30,000 tickets a month than the model bill in the cost section below, so price any tool at your own volume. The model bill is not the full cost of building, though: add the engineering and upkeep above.

Don't build if

  • An existing tool covers 70% or more of your need.
  • Your use case is generic (cold email, scheduling, support) and a bought tool costs less at your volume, engineering and upkeep included.
  • You are doing it because "AI agents are hot," not because of a real problem.
  • You don't have an engineer with production LLM experience on the team.
  • You can't articulate what success looks like in one sentence.

Build if

  • No existing agent solves your specific case after a real search.
  • The integrations you need don't exist as off-the-shelf tools.
  • The agent IS the product or a core competitive moat.
  • You need data flows that off-the-shelf tools can't support.
  • Your security requirements rule out third-party hosted agents.

Workflow or agent? The distinction that saves weeks

From Anthropic's research: workflows are systems where LLMs and tools are orchestrated through predefined code paths. Agents are systems where LLMs dynamically direct their own processes and tool usage. The implication is significant. If your problem can be solved with a fixed sequence of LLM calls and known tools, you do not need an agent. You need a workflow, which is dramatically simpler to build and maintain.

Workflow example

Process incoming support emails: extract sentiment, classify into one of 5 categories, look up the customer record, draft a templated reply.

Same 4 steps every time. Predictable. Easy to debug. Cheap to run. Don't build an agent for this.

Agent example

Triage support tickets where the LLM decides: do I have enough info? Should I check the order system, the billing system, the shipping API, or escalate?

Variable paths. The LLM picks tools based on what it discovers. This needs a real agent.

We'll use the second example, a customer support triage agent, as the running case throughout this guide.

The 4 components every agent has

Strip away the framework hype and every agent is just these four pieces. Get any one wrong and the whole thing fails.

1

Reasoning engine

The LLM that decides what to do next. Claude, GPT, or Gemini.

2

Tools

Functions the LLM can call to take action: query a DB, send an email, hit an API.

3

Memory

What persists between calls: conversation history, prior tool results, long-term facts.

4

Reasoning loop

The orchestration: receive input, plan, call tool, evaluate, repeat, with validation and limits.

Decision 1: Pick your model

The table is an illustrative selection of a few models, not a ranking, and it leaves out other providers such as OpenAI. It shows standard API prices per million tokens (input / output), as listed on each provider's own pricing page on 4 November 2026. Providers release new models and change prices often, so check the current pricing page before budgeting. The recommendation: start with the cheapest model that passes your evals. Many teams default to flagship models out of habit and spend more than they need to.

ModelPricing (per 1M tokens)Best for
Claude Haiku 5.5$0.10 / $0.50Low-cost starting point for high-volume tool use. Price for prompts up to 100K tokens ($0.50 / $2.50 above that).
Claude Sonnet 5.5$2 / $10Mid-tier Claude model for harder reasoning steps.
Claude Opus 5.5$4 / $20Larger Claude model for long-running agentic work.
Gemini 3.8 Flash$0.75 / $3.75Price through December 31, 2026. Google lists $1.50 / $7.50 from January 1, 2027.
Gemini 4 Argon$2 / $10Google's flagship reasoning model. Price through November 30, 2026. Google lists $4 / $20 from December 1, 2026. Not available on the free tier.

Practical rule: build your eval set first, run it on the cheapest viable model, then upgrade only when you hit a wall. Many agent tasks do not need flagship reasoning.

Decision 2: Design your tools

Tools are what turn an LLM into an agent. The model needs three things to use a tool reliably: a clear name, a description that explains when to use it (not just what it does), and a strict input schema. Here's a real tool definition for our customer support agent, using Anthropic's tool use format, but the structure is similar across providers:

import Anthropic from "@anthropic-ai/sdk"

const anthropic = new Anthropic() // reads ANTHROPIC_API_KEY

const tools: Anthropic.Tool[] = [
  {
    name: "lookup_order",
    description: "Look up a customer order by email or order ID. Use this when the customer asks about delivery status, items in their order, or refund eligibility.",
    input_schema: {
      type: "object",
      properties: {
        email: { type: "string", description: "Customer email" },
        order_id: { type: "string", description: "Order ID like ORD-12345" }
      },
      required: []
    }
  },
  {
    name: "check_shipping_status",
    description: "Get real-time shipping status from the carrier. Use AFTER lookup_order has confirmed the order exists and shipped.",
    input_schema: {
      type: "object",
      properties: {
        tracking_number: { type: "string" }
      },
      required: ["tracking_number"]
    }
  },
  {
    name: "escalate_to_human",
    description: "Hand off to a human agent. Use when the issue involves refunds over $500, account security, or the customer explicitly asks for a person.",
    input_schema: {
      type: "object",
      properties: {
        reason: { type: "string", description: "Why escalation is needed" },
        priority: { type: "string", enum: ["low", "medium", "high"] }
      },
      required: ["reason", "priority"]
    }
  }
]

Three tool design rules that matter:

Decision 3: Memory

Memory has three tiers and many teams overbuild this. Use the simplest tier that meets your needs. Vector DBs are not free.

Tier 1: In-context

When: Single conversation, fits in the context window.

How: Just pass message history with each call. No infrastructure. Works for most agents under 10 turns.

Tier 2: Session-persistent

When: Multi-session conversations or workflows that span hours/days.

How: Store messages array in Postgres or Redis keyed by session ID. Retrieve on next call. Cheap, fast, deterministic.

Tier 3: Long-term semantic

When: Agent needs to recall facts across thousands of past interactions.

How: Vector DB (Supabase pgvector, Pinecone, Weaviate) with embedding search. Adds complexity and cost. Only build this when Tier 2 demonstrably fails.

Decision 4: The reasoning loop

The loop is where many agents fail. Here's a minimal skeleton. It shows the iteration cap, a basic input check, and explicit error handling. Skip any of these and your agent will eventually hit a state that costs you real money. It is not production-ready as written: it only checks that the tool exists and required fields are present. Before launch, add full schema validation, a per-conversation cost cap, loop detection, context summarization and the is_error flag on failed tool results, as the failure modes and checklist below explain:

// Your own functions, one for each tool in the tools array
const handlers: Record<string, (input: Record<string, unknown>) => Promise<string>> = {
  lookup_order: lookupOrder,
  check_shipping_status: checkShippingStatus,
  escalate_to_human: escalateToHuman
}

async function executeToolSafely(name: string, input: unknown): Promise<string> {
  const tool = tools.find(t => t.name === name)
  const handler = handlers[name]
  if (!tool || !handler) return "Error: unknown tool " + name

  const params = input as Record<string, unknown>
  const missing = (tool.input_schema.required ?? []).filter(field => params[field] == null)
  if (missing.length > 0) return "Error: missing " + missing.join(", ")

  try {
    return await handler(params)
  } catch (err) {
    return "Error: " + String(err)
  }
}

async function runAgent(userMessage: string) {
  const messages: Anthropic.MessageParam[] = [{ role: "user", content: userMessage }]
  const MAX_ITERATIONS = 10
  let iterations = 0

  while (iterations < MAX_ITERATIONS) {
    const response = await anthropic.messages.create({
      model: "claude-haiku-5-5",
      max_tokens: 1024,
      system: "You are a customer support triage agent. Use the tools to check orders and shipping. Escalate to a human when a tool description says to.",
      tools,
      messages
    })

    // Agent finished - return the final message
    if (response.stop_reason === "end_turn") {
      return response
    }

    // Agent wants to call one or more tools (it can ask for several at once)
    if (response.stop_reason === "tool_use") {
      const toolUses = response.content.filter((c): c is Anthropic.ToolUseBlock => c.type === "tool_use")
      if (toolUses.length === 0) throw new Error("Expected tool_use block")

      // Run every requested tool. executeToolSafely checks the tool exists and required fields are present.
      // Every tool_use needs its own tool_result, or the next request is rejected.
      const results: Anthropic.ToolResultBlockParam[] = []
      for (const toolUse of toolUses) {
        const result = await executeToolSafely(toolUse.name, toolUse.input)
        results.push({ type: "tool_result", tool_use_id: toolUse.id, content: result })
      }

      // Append assistant turn + all tool results, then loop
      messages.push({ role: "assistant", content: response.content })
      messages.push({ role: "user", content: results })
    } else {
      // max_tokens, refusal or any other stop: fail loudly instead of re-sending the same request
      throw new Error("Agent stopped unexpectedly: " + response.stop_reason)
    }

    iterations++
  }

  throw new Error("Agent exceeded max iterations - possible loop")
}

The 5 production failure modes

Expect to run into these. Plan for them on day one. Bolting them on after launch is a much harder retrofit.

1. Hallucinated tool calls

Symptom: Agent invents tool names or parameters that don't exist in your schema.

Fix: Validate every tool call against your schema before execution. On failure, return a structured error to the model so it can self-correct on the next iteration.

2. Infinite loops

Symptom: Agent calls the same tool over and over, or oscillates between two tools, never reaching end_turn.

Fix: Hard iteration cap (10-15 is reasonable for most agents). Track tool call patterns and break loops if the same call repeats more than 3 times.

3. Parameter type errors

Symptom: Agent passes a string where you expected a number, or omits a required field.

Fix: Strict JSON schema validation on every tool call. Coerce types where safe, error explicitly otherwise. Always include type constraints in your schema descriptions.

4. Context window blow-out

Symptom: After 20+ turns the agent slows to a crawl and costs spike. Worst case: it errors out from exceeding the context limit.

Fix: Summarize old messages once history exceeds a threshold (e.g. > 50K tokens). Replace the oldest 20 messages with a 200-token summary. Lose detail, keep coherence.

5. Cost runaway

Symptom: A single misbehaving conversation runs up a large API bill before you notice.

Fix: Per-conversation cost cap that hard-stops the loop. Real-time cost tracking exposed to whoever's on call. Alerting at $X per session.

Cost math: what you'll actually pay

Here's a worked example for our customer support triage agent at production scale. The volumes are illustrative assumptions, and the per-token prices are the standard rates on each provider's pricing page on 4 November 2026. Assumptions: 1,000 tickets per day, average 2 tool calls per ticket (3 LLM calls: one to choose each tool and one for the final reply), roughly 4,000 input tokens (system prompt + tool definitions + conversation context) and roughly 500 output tokens per LLM call.

Per ticket: 3 LLM calls x (4,000 input + 500 output tokens)
= 12,000 input + 1,500 output tokens per ticket

1,000 tickets/day = 12M input + 1.5M output tokens/day
Monthly (30 days) = 360M input + 45M output tokens

Claude Haiku 5.5: (360 x $0.10) + (45 x $0.50) = $58.50/mo
Gemini 3.8 Flash: (360 x $0.75) + (45 x $3.75) = $438.75/mo through December 31, 2026 ($877.50/mo at the January 1 rates)
Claude Sonnet 5.5: (360 x $2) + (45 x $10) = $1,170/mo
Gemini 4 Argon: (360 x $2) + (45 x $10) = $1,170/mo through November 30, 2026 ($2,340/mo at the December 1 rates)
Claude Opus 5.5: (360 x $4) + (45 x $20) = $2,340/mo

Both Gemini prices above change soon, so if you are starting a 6 to 12 week build today, budget on the later rates. Two things this calculation also hides: prompt caching can cut input costs sharply if you reuse a stable system prompt (for the Claude models in the table above, Anthropic bills cache reads at between a tenth and a twentieth of the base input price), and context growth on long conversations can multiply these numbers if you don't implement summarization. Budget conservatively. Your real cost will likely be higher than your napkin math the first time.

Pre-deploy checklist

Before you put this in front of users, every box below should be ticked. Skipping any of these is how teams end up with surprise bills or angry customers.

When you're done iterating

Agents are never "finished." But there's a point where further tuning has diminishing returns and you should stop touching it. You're done iterating when: your eval set passes consistently above your target threshold, the failure modes you see in production are the same as the failure modes you see in eval (no surprises), per-conversation cost is stable and predictable, and the agent can run unattended for a week without you needing to check on it.

Past that point, your time is better spent building the next agent than tuning this one.

Frequently Asked Questions

What is the difference between an AI workflow and an AI agent?

A workflow is a predefined sequence where an LLM is called at fixed steps with predictable inputs and outputs. An agent is a system where an LLM dynamically decides which tools to call and in what order until it accomplishes a goal. Anthropic published this distinction in their December 2024 research note "Building Effective Agents." Many teams should build workflows first and only graduate to true agents when the workflow cannot handle the variability they actually encounter.

When should I build an AI agent vs use an existing one?

Build only when no existing agent solves your specific use case, the integrations you need do not exist as off-the-shelf tools, or you are building proprietary functionality that becomes a competitive moat. Buy when an existing agent covers 70% or more of your needs, your use case is generic (cold email, customer support, scheduling) and a bought tool costs less at your volume, or the engineering cost would exceed two years of subscriptions to an existing tool.

How much does it cost to run an AI agent in production?

Cost depends on three variables: model pricing, average tokens per call, and call volume. A customer support triage agent processing 1,000 tickets per day with 4,000 input tokens and 500 output tokens per call is the worked example in this guide. Multiply your monthly input and output tokens by the current per-million-token price of each model, as listed on the pricing page of its provider. Most teams underestimate context window growth on multi-turn agents, and costs can rise sharply in long conversations if you do not actively manage context.

What is the most common reason AI agents fail in production?

A frequent failure mode is hallucinated tool calls, where the agent invents tool names or parameters that do not exist in your schema. The fix is strict schema validation before execution, retry with corrective feedback, and a hard iteration cap to prevent infinite loops. Other common failures include context window blow-out on long conversations, parameter type errors, and unbounded cost from runaway loops.

When should I use multiple agents instead of one?

Use multiple agents when a single agent cannot handle all the tasks reliably within one context window, when tasks are genuinely parallelizable and can run simultaneously, or when different tasks require meaningfully different system prompts and tool sets that would conflict if combined. Many teams reach for multi-agent architecture too early. A well-built single agent with a clear scope and good tool definitions handles many production use cases. Only add a second agent when you have hit a concrete limitation of the first, not in anticipation of limitations you have not encountered yet.

AI Coding Agents

Tools to help you build →

Multi-Agent Orchestration

When one agent isn't enough →

How to Evaluate an Agent

If you decide to buy →

No-Code Builders

Skip the engineering →

See our editorial methodology. Model prices checked against each provider's pricing page on 4 November 2026. Confirm current rates before budgeting.

AI Agent Price & Rating Tracker

Free, every two weeks. Price changes, new agent launches, acquisitions, and rating updates across the AI agent market. Verified against live vendor data, not vendor marketing.

No spam. Unsubscribe anytime. We never share your email. Read past issues

Sources & References

  1. 1.
  2. 2.
  3. 3.
    Building Effective Agents — Anthropic, 2024