How do AI agents work? Tools, memory and reasoning explained

An AI agent works by running a language model in a loop. On each turn the model reads its instructions, the goal and everything gathered so far, then picks one next action: answer, or ask for a tool such as an order lookup, a calendar check or an email send. Ordinary software carries out that request and feeds the result back, and the loop repeats until the job is done, a step limit is reached or a person is needed. What people call memory is whatever the software chooses to send back in on the next turn, because the model itself keeps nothing between calls.

That design fact explains most of the odd behaviour owners report: agents that forget, repeat themselves, or get nine steps right and the tenth wrong. Our guide to agentic AI covers what the category is and where it pays. This page opens the bonnet.

One turn of the loop

Take a customer email asking to move a delivery to Friday. Here is what happens inside a typical agent, in order.

1. The software assembles a request: instructions, the email, and a menu of tools, each with a name, a description and the inputs it expects.
2. The model replies with a decision. Either it writes an answer, or it returns a tool request, for example find_order with the customer's email address filled in.
3. Your code runs the tool. It checks the request is allowed and queries the order system.
4. The result goes back in, added to the conversation, and the whole lot is sent to the model again.
5. Repeat. Next might come check_delivery_slots, then draft_reply, until the model gives a final answer or a limit stops it.

Anthropic, which makes the Claude models, puts the first point bluntly in its developer documentation: "The model never executes anything on its own." Its engineers describe agents as "LLMs using tools based on environmental feedback in a loop". Both lines are a vendor describing its own product. The pattern itself, where the model asks and your code acts, is what developers call function calling.

Tools are where the permissions live

Because your code runs every tool, that code is where limits go. The model can ask for a refund. Whether one happens depends on the tool: a cap, a check against the order value, or a pause for a manager.

So split tools by consequence. Reading a record is cheap to get wrong. Drafting a reply is cheap too, because nobody has seen it yet. Sending, paying and deleting are not, and those are the tools that sit behind an approval gate. Our page on AI agents for business sets out how to scope that access role by role.

Memory is four different things

Vendors use one word for four mechanisms that fail in different ways. Ask which one a product means.

The first row has research behind it. A Stanford-led study published in 2024 found that model performance "significantly degrades when models must access relevant information in the middle of long contexts", and that this held even for models built for long inputs. A bigger context window isn't a cure. Sending less, and putting what matters near the start or end, works better.

Reasoning happens one step at a time

An agent doesn't hold a plan in its head. On each turn it predicts the most sensible next action from the text in front of it. Asking the model to write a plan first helps, but that plan is just more text in the context. When a tool returns an error, a well-built agent tries something else. A badly built one asks again.

That's why stopping rules belong in the software, not the prompt. Anthropic's own guidance recommends stopping conditions such as a maximum number of iterations. Set a cap on turns, on tool calls and on spend, and route anything that hits a cap to a named person with the transcript attached.

The reliability maths every buyer should run

Each turn is a fresh chance to go wrong, and the chances multiply. The table assumes every step succeeds independently at the same rate. Real systems don't quite, but the shape holds.

Twenty-seven steps isn't an arbitrary column. In TheAgentCompany benchmark from Carnegie Mellon and Duke, the strongest agent took almost 27 steps per task on average and completed 30% of the tasks on its own. Our AI employees page breaks those results down by job.

Consistency is the other half. Sierra's τ-bench, a peer-reviewed benchmark of agents serving simulated customers under company policies, gave GPT-4o about 61% on retail tasks and about 35% on airline tasks. Asked to get the same retail task right on all 8 attempts, it managed about 25%. Sierra sells customer service agents, though these figures come from its peer-reviewed paper, not its marketing.

Models are improving fast. METR, an AI evaluation group, found the length of task frontier models can finish half the time has doubled about every 7 months. For o3 that horizon was software work taking a person around 110 minutes. Half the time is a coin toss, not an operating standard for invoices or customer replies.

What this means when you design one

• Make the model decide fewer steps. Anything predictable (send the confirmation, update the status field) belongs in fixed code. The difference is laid out in our note on agents versus RPA and the agentic workflows guide.
• Check after each action. Read the record back after writing it; a tool reporting success isn't proof.
• Treat every tool result as untrusted text. An email or web page fed into the context can contain instructions aimed at the model. The UK's National Cyber Security Centre warned in December 2025 that prompt injection "may never be totally mitigated in the way SQL injection attacks can be", so limit what any one tool can do.
• Run it in shadow mode first, scoring its output against staff before it sends anything.

Common questions

Pick one process and write down every step a person takes today, including the lookups. If the list runs past ten decisions, that number tells you how much of it should be fixed code and how much needs a model. Send the list, with roughly how many cases arrive each month, through the contact form under Business automation; after an informal scoping chat we'll come back with a price range for the build.