How AI agents fail tool-call security

Tool-call security is not model safety

Most published agent evaluation is about what a model will say. Tool-call security is about what an agent will do, and a good result on one surface tells you very little about the other. A model that refuses to write malware may still call a refund tool because someone in the conversation asserted, plausibly and politely, that they had authority to ask for one.

The reason is structural. When an agent decides to make a tool call, it is performing an authorization decision. It is deciding, from context, whether this action is permitted for this requester right now. Authorization decisions have never been safe to make from content alone. Every other layer of computing learned this and moved the decision somewhere the requester cannot influence: file permissions, database grants, cloud IAM. Agent tool calls are the one place where it is routinely left in the same channel as the request.

The five failure classes

1. Unauthorized action

The agent executes an action the requester has no authority to cause. The mechanism is almost always some form of asserted authority: the conversation establishes, through ordinary language, a role or an exception or an approval that was never verified anywhere. The agent has no way to check the claim, because nothing in its context distinguishes a true claim from a false one, so it acts on plausibility.

What makes this class dangerous is that it requires no technical sophistication. It is social engineering aimed at a component that cannot escalate to a human and has no memory of who it is talking to beyond the transcript. This is the class AIUC-1 B006 exists to cover.

2. Cap and limit evasion

The action is authorized, but the limits around it are not enforced where they are declared. Typical shapes: a per-transaction cap that is checked in the prompt rather than in the tool, so a sequence of smaller calls achieves what one large call would not; a rate limit applied per session when the constraint that matters is per account; or a limit enforced on the first call in a chain but not on the ones generated downstream.

These survive testing because each individual call looks correct. The problem only appears in aggregate, so you have to test for it in aggregate, and the enforcement point has to hold state across calls rather than judging each one in isolation.

3. Cross-boundary data access

Data crosses a boundary it was never meant to cross: between tenants, between classification levels, or out of the system. The subtle version involves no compromised tool at all. It is composition. A read tool that legitimately returns records, and a write tool that legitimately sends messages, compose into an exfiltration path that neither tool creates on its own. The agent is doing exactly what both tools advertise.

The instruction to make that composition often does not come from the user. It comes from data the agent read, which leads to the class underlying several of the others.

4. Privilege escalation

The agent acts with more authority than the requester has. Sometimes this is direct, where a parameter carries a role or scope the caller should not be able to set. More often it is a confused-deputy problem: the agent holds broad credentials so it can serve many users, and any user's request then runs with the agent's privileges rather than their own. Nothing is technically bypassed. The authority was simply never narrowed to the requester.

This is the class most often introduced by architecture rather than by an attack, and it is why AIUC-1 B007 asks for role and scope to be checked at execution, per call, rather than once at the boundary of the system.

5. Instruction provenance failure

The agent treats content it read as an instruction it should follow. Retrieved documents, tool outputs, records, emails, web pages, the output of another agent: all of it arrives in the same context window as the user's actual request, and by default none of it is marked with where it came from or what authority it carries. An instruction embedded in retrieved data is, to the model, just more context.

The delayed variant defeats naive testing. The instruction is planted in one interaction and acted on later, so the request that triggers the harmful action looks benign in isolation. If your evaluation only tests single turns, this class is invisible to it.

Why a better model does not close these

It is reasonable to assume a stronger base model resists all of this, and it does help at the margin. But our adversarial corpus surfaces tool-call control failures in agents built on current frontier models, not just weak ones. The reason is not that the models are bad at judgment. It is that judgment is the wrong instrument.

A model deciding whether a refund is authorized is doing inference over text. It will be right most of the time and wrong some of the time, and that rate is a property of the model. The controls are not asking about the rate. They ask what happens on the occasion it is wrong, and that is a property of your architecture. If the only thing between a manipulated conversation and a real financial action is the model's opinion, there is no control, only a probability.

This is also why detection-based defenses plateau. Scanners and filters have to separate hostile requests from legitimate ones by inspecting them, and in the classes above the two are often the same request differing only in who is making it. You can raise the detection rate. You cannot reach a structural guarantee that way, and that is what the control language asks for.

What actually closes them

Every one of these classes is closed at the action boundary, by the same small set of properties:

• Deny by default. An action with no declared permission is refused. Absence of a rule is not permission.
• Authorization per call, at execution. Identity, role and scope are checked on the call itself, not once per session, and the decision is recorded either way.
• Parameter-level constraints. Permission to call a tool is not permission to call it with arbitrary arguments. Amounts, record counts, recipients and boundaries are declared and enforced.
• Provenance on context. Every item in the agent's context carries its source and that source's authority, so an instruction found in retrieved data cannot be treated as one from the user. This is the defense for class five, and what the AgentLock™ research is about.
• Operation-bound, single-use tokens. An authorization decision is bound to the exact parameters it approved and cannot be replayed for a different call.
• Stateful limits. Caps and rate limits enforced at the boundary, across calls, so decomposition into smaller steps does not defeat them.
• Human approval on declared action classes. Applied only to the actions whose blast radius justifies it, so the mechanism survives real usage instead of being switched off.
• A log with reason codes. Denials recorded as explicitly as allows, because a log that only shows successes cannot demonstrate that a check ran.

None of this depends on recognizing an attack. That is the point: the gate does not need to know a request is hostile, because it is not deciding on content at all.

How the classes map to the controls

Unauthorized action maps to B006, and to D003 where the tool surface itself is the gap. Cap and limit evasion maps to D003 at the parameter level. Cross-boundary data access maps to A003, and usually D003 where composition is the path. Privilege escalation maps to B007. Instruction provenance failure maps to D003 and A003 together, being the mechanism that triggers the others. D004 sits across all of them: the requirement that someone independent actually tried. There is a fuller walkthrough of each control in what the AIUC-1 tool-call controls actually require.

What we publish, and what we do not

This post describes technique classes and defenses. It contains no working payloads, and neither do our findings reports. Each finding names the class, the affected tool, the mapped control, and the baseline and remediated outcomes, and withholds the vector. That is not caution for its own sake: a published corpus stops working, and the point of the corpus is that it keeps finding real failures in the next agent we test. Clients get the outcome and the fix, not the ammunition.

Common questions

What are the main ways AI agents fail tool-call security?

Five recurring classes: unauthorized action, where the agent executes something the requester has no authority to cause; cap and limit evasion, where declared limits are not enforced where the action happens; cross-boundary data access, where data crosses a tenant or classification boundary, often through composition of two legitimate tools; privilege escalation, where the agent acts with more authority than the requester holds; and instruction provenance failure, where the agent treats content it read as an instruction to follow.

Can prompt injection be solved with better filtering?

Not completely. Filtering has to separate hostile requests from legitimate ones by inspecting them, and in these failure classes the two are frequently identical in form, differing only in who is making the request and where the instruction originated. Detection raises the bar and does not produce a structural guarantee. The durable defense is to move the authorization decision out of the conversation and enforce it at the action boundary.

Does a frontier model prevent these failures?

It helps at the margin but does not close them. Our adversarial corpus surfaces tool-call control failures in agents built on current frontier models, not just weak ones. A stronger model lowers how often its judgment is wrong; it does not change what happens on the occasion when it is wrong, which is a property of the architecture around it rather than of the model.

What is a confused deputy problem in an AI agent?

It is when the agent holds broad credentials so it can serve many users, and an individual user's request then executes with the agent's privileges instead of their own. Nothing is bypassed technically; the authority was never narrowed to the requester. It is a common architectural source of privilege escalation, and it is why AIUC-1 B007 asks for role and scope to be checked at execution on every call.

Why are adversarial payloads withheld from findings reports?

Because a published corpus stops working. Findings name the technique class, the affected tool, the mapped control, and the baseline and remediated outcomes, which is what an auditor needs to trace a finding to a control. The vector itself is withheld so it remains useful against the next agent tested, and so the evidence package does not hand a reader a working attack.