What makes AI agents a different security problem
A traditional application has a defined set of operations it can perform. An AI agent has a reasoning engine that can dynamically determine what to do and how — using whatever tools it has been granted access to. This creates a security problem that has no direct equivalent in traditional software.
Autonomous decision-making: The agent determines its own action sequence to achieve a goal. A developer can specify the goal and provide the tools, but the agent decides which tool to call, with what parameters, and in what order. Unexpected or adversarially-influenced decisions can produce significant unintended consequences.
Broad tool access: Production agents are increasingly given access to email, file systems, databases, APIs, web browsers, and code execution environments. Each tool is a potential impact surface. An agent with write access to a database and send access to email can do significant damage if its behavior is compromised.
Persistent state: Unlike a chatbot that processes one message and produces one response, agents maintain state across steps and across sessions. Actions taken early in an agent run can have consequences that persist long after the run completes. Irreversible actions — sending emails, committing transactions, deleting records — cannot be undone.
Indirect control surface: Agents are influenced not only by their system prompt but by all the content they process — documents they retrieve, web pages they browse, emails they read. This creates a large indirect attack surface, particularly through prompt injection.
The AI agent threat model
Security analysis of AI agents typically identifies four categories of threat:
1. Prompt injection and indirect manipulation
An attacker embeds malicious instructions in content the agent will process — a document the agent is asked to summarize, a web page it browses, an email it reads. The injected instructions redirect the agent's behavior, potentially causing data exfiltration, unauthorized actions, or privilege escalation. Because agents process large volumes of content from diverse sources, indirect prompt injection is the primary external attack vector.
2. Privilege escalation and scope creep
An agent given access to one system may be able to escalate its access through the actions it takes — discovering credentials, following links to connected systems, or using tokens and keys encountered in the data it processes. Without strict scope enforcement at the infrastructure layer, an agent's effective access may be significantly broader than its intended access.
3. Data exfiltration
An agent with access to sensitive data and an outbound communication channel — email API, webhook, HTTP call — can be directed to exfiltrate data. This can happen through successful prompt injection, through misconfigured tool permissions, or through a compromised agent supply chain (a compromised model, a malicious tool, or a tampered prompt template).
4. Irreversible action execution
Agents taking real-world actions — submitting transactions, sending communications, modifying records, deploying code — create risk through irreversibility. Unlike a human who can recognize when an action is outside their authority and pause, an agent will generally proceed if it has the technical capability. Lack of human approval gates for high-impact actions is one of the most common AI agent security gaps in production deployments.
Core AI agent security controls
Least-privilege tool access
Each agent should be granted the minimum tool access necessary to accomplish its task. An agent that needs to read emails should not have send permissions. An agent that needs to query a database should not have write permissions. Restricting what an agent can do at the infrastructure layer is the most durable security control — it limits the blast radius of any failure, injection, or misconfiguration regardless of cause.
This is the AI equivalent of the principle of least privilege in traditional access control, applied to agent capabilities rather than user permissions.
Runtime policy enforcement
A policy enforcement layer sits between the agent and the tools it calls, evaluating every proposed action against a defined policy before execution. The policy can specify:
- Which tools the agent is permitted to call
- What parameter values are within bounds (e.g., which endpoints the agent may call, which data categories may be submitted to external services)
- Which actions require human approval before proceeding
- What data classifications are permitted to leave the organization's systems
Runtime policy enforcement operates at the infrastructure layer — it does not depend on the agent's prompt or its self-reported intent. This makes it effective even when the agent's behavior is influenced by prompt injection or other adversarial manipulation.
Human approval gates
For high-risk actions — actions that are irreversible, have significant scope, or involve sensitive data — require explicit human approval before the agent proceeds. Common categories for approval gates include:
- External communications (sending emails, posting to external APIs)
- Data modification (writing to databases, modifying files)
- Financial actions (initiating transfers, committing transactions)
- Privilege-escalating actions (creating accounts, modifying permissions)
Human-in-the-loop (HITL) approval does not eliminate the value of autonomous agents — it targets approval requirements specifically at actions where human judgment materially reduces risk.
Tamper-evident audit logging
Every agent action should be logged at the infrastructure layer with: the action requested, the tool called, the parameters submitted, the policy decision applied, and the outcome. Logs should be tamper-evident and retained for the full audit period.
Infrastructure-layer logging is important because agent-layer logging (a log the agent itself produces) can be manipulated by the same adversarial inputs that affect the agent's behavior. An independent log that the agent does not control is a more reliable basis for incident investigation and compliance evidence.
Agent identity and isolation
Each agent instance should operate with a scoped, auditable identity — not shared credentials or ambient access inherited from the deploying user. Agent identity enables: attribution of actions to specific agent instances; access control based on agent role rather than deploying user; and revocation of a specific agent's access without affecting other systems.
In multi-agent architectures, agent-to-agent communication should be authenticated and authorized — an orchestrating agent should not be able to instruct a subagent to perform actions beyond the subagent's defined authorization.
AI agent security vs. LLM security
| LLM Security | AI Agent Security | |
|---|---|---|
| Focus | Model-layer attacks and vulnerabilities | Runtime behavior of autonomous systems |
| Primary threats | Jailbreaking, training data extraction, model inversion | Prompt injection, privilege escalation, data exfiltration, irreversible actions |
| Control layer | Input/output filtering, content classification | Tool call interception, policy enforcement, approval gates |
| Audit scope | Prompt and response content | Full action trace: tools called, parameters, policy decisions, outcomes |
| Applies to | Any LLM deployment | Agentic workflows with tool access and real-world impact |
LLM security and AI agent security address overlapping but distinct concerns. An organization using an AI agent in production needs both: model-level controls to address prompt attacks, and agent-level controls to govern what the agent does with the access it has.
Where AI agent security sits in the stack
A complete AI agent security stack includes three layers:
1. Input controls — What content reaches the agent and how it is validated before processing. Includes prompt injection detection, content classification, and source verification.
2. Runtime controls — What the agent can do during execution. Includes least-privilege tool access, policy enforcement at each tool call, and human approval gates for high-risk actions.
3. Output and audit controls — What the agent's actions produce and how they are recorded. Includes tamper-evident logging, output monitoring, and anomaly detection.
Most production agent deployments have input controls (via the model provider's content filtering) and partial audit controls (via provider logs). The runtime control layer — independent policy enforcement at the action layer — is the most common gap.
Related: Prompt Injection · Agentic AI Risk · LLM Security · AI Governance
