Artificial intelligence is becoming part of everyday software. Chatbots answer questions, AI assistants summarize documents, and AI agents can interact with websites, APIs, and business tools.
However, AI systems can face a unique security problem: prompt injection.
Instead of attacking a server directly, an attacker can manipulate the instructions that an AI system receives. As a result, the AI may produce an unexpected response, reveal information, or perform an action that the developer never intended.
OWASP currently lists Prompt Injection as LLM01:2025, the first risk in its Top 10 for Large Language Model applications.
What Is Prompt Injection?
Prompt injection happens when someone provides instructions that change an AI system’s intended behavior.
The attacker does not necessarily need to break into the AI model itself. Instead, they try to influence how the model interprets instructions and data.
For example, an AI application might normally follow this rule:
Summarize the document provided by the user.
An attacker could place additional instructions inside the document. When the AI reads that content, it may treat those instructions as something it should follow.
Therefore, the problem comes from the way AI systems process instructions and external content.
NIST describes prompt injection as an attack that exploits the combination of untrusted input with instructions created by a trusted application designer.
Why Is Prompt Injection Different From Traditional Hacking?
Traditional attacks often target software vulnerabilities.
For example, attackers may exploit:
- Weak passwords
- Unpatched software
- SQL injection
- Buffer vulnerabilities
- Misconfigured servers
Prompt injection works differently.
Here, the attacker targets the AI’s interpretation of language.
An attacker may provide carefully designed instructions that conflict with the application’s original purpose. Consequently, the model may prioritize or interpret the malicious content in an unexpected way.
This makes prompt injection an important concern for developers building AI-powered applications.
How Does a Prompt Injection Attack Work?
A simple AI application might contain three elements:
System instructions → User input → AI model
The system instructions define the AI’s role.
The user input provides the request.
Finally, the model processes both types of information.
The security problem appears when the application does not clearly separate trusted instructions from untrusted content.
An attacker can then introduce additional instructions through the input.
For example, a customer-support AI might receive a message containing malicious instructions. If the application gives the model access to private systems, the manipulated model could potentially attempt actions beyond its intended task.
OWASP notes that prompt injection can lead to sensitive information disclosure, unauthorized access to functions, and manipulation of important decisions.
Direct Prompt Injection
Direct prompt injection happens when the attacker directly interacts with the AI system.
For example, a user may intentionally enter instructions designed to change the model’s behavior.
The attacker might try to:
- Override the application’s instructions
- Request hidden information
- Manipulate the model’s response
- Bypass safety controls
- Influence a connected function
This attack is relatively easy to understand because the malicious input comes directly from the user.
However, direct attacks are only one part of the problem.
Indirect Prompt Injection
Indirect prompt injection can be more difficult to detect.
Instead of sending the malicious instructions directly to the AI, an attacker places them inside content that the AI will later process.
That content could appear in:
- A webpage
- A document
- An email
- A support ticket
- A code repository
- A product review
- A database record
For example, imagine an AI assistant that summarizes webpages.
An attacker creates a webpage containing normal-looking information along with hidden instructions intended for the AI.
Later, a user asks the assistant to summarize that webpage.
The AI reads the malicious content as part of the webpage. Consequently, the embedded instructions may influence its behavior.
OWASP specifically identifies external webpages and files as potential sources of indirect prompt injection.
Why Connected AI Systems Face Greater Risk
A basic chatbot may only generate text.
However, modern AI applications can do much more.
An AI agent may have permission to:
- Send emails
- Search databases
- Access files
- Call APIs
- Create tickets
- Update records
- Use cloud services
- Perform automated tasks
Therefore, prompt injection becomes more serious when an AI system has access to external tools.
A manipulated response might then influence a connected application.
The AI does not need to be completely compromised for this to create a security problem. Instead, excessive permissions can increase the impact of a successful manipulation.
What Can a Successful Prompt Injection Cause?
The impact depends on how the AI application works.
Potential consequences include:
Sensitive Information Exposure
An attacker may attempt to make the AI reveal information that it should not provide.
System Prompt Leakage
Attackers may try to discover hidden instructions used to configure an AI application.
Unauthorized Actions
If an AI agent can call external tools, manipulated instructions could influence those actions.
Incorrect Decisions
An attacker may manipulate the information presented to the AI and influence its output.
Data Manipulation
An AI connected to business systems could potentially contribute to incorrect changes if developers do not enforce proper controls.
Social Engineering
Attackers can also use manipulated AI outputs to influence users.
OWASP warns that prompt injection can affect sensitive information, connected functions, and critical decision-making.
Prompt Injection vs Jailbreaking
People often use these terms interchangeably, but there is a useful distinction.
Prompt injection is the broader concept. It involves manipulating an AI application’s behavior through crafted input.
Jailbreaking focuses more specifically on getting an AI model to bypass its safety restrictions.
In other words, jailbreaking can be considered a type of prompt injection.
The important point is that both techniques attempt to influence how an AI system follows its intended instructions.
Can Developers Completely Prevent Prompt Injection?
Unfortunately, there is no single security control that guarantees complete prevention.
Large language models process natural-language instructions and data together. As a result, developers cannot rely on one prompt or filter to solve every possible attack.
OWASP recommends reducing the impact of prompt injection through multiple security layers rather than depending on a single defense.
How Can Developers Reduce Prompt Injection Risks?
1. Separate Instructions From External Data
Applications should clearly distinguish trusted instructions from untrusted content.
For example, content retrieved from a webpage should not automatically receive the same trust as the application’s system instructions.
This separation can reduce the influence of malicious external content.
2. Limit AI Permissions
An AI application should receive only the access it needs.
For example, an AI assistant that summarizes documents does not need permission to delete files.
This follows the principle of least privilege.
If an attacker successfully manipulates the AI, limited permissions can reduce the potential damage.
3. Validate AI Outputs
Developers should not blindly trust AI-generated output.
Instead, applications should validate important outputs before using them.
This is especially important when AI output reaches:
- Databases
- APIs
- Financial systems
- Code execution environments
- Administrative tools
4. Require Human Approval for High-Risk Actions
Some actions should require explicit human confirmation.
For example, an AI agent could prepare an email but ask the user to approve it before sending.
Similarly, an agent could prepare a database change without automatically executing it.
OWASP recommends human approval for high-risk operations as one mitigation against prompt injection and excessive agency.
5. Treat External Content as Untrusted
Websites, files, emails, and documents can contain instructions designed to influence an AI.
Therefore, applications should treat external content as untrusted data.
This approach becomes especially important for AI systems that automatically retrieve information from the internet.
6. Monitor AI Activity
Security teams should monitor unusual AI behavior.
For example, an AI assistant that suddenly requests access to a sensitive database may deserve investigation.
Monitoring can help organizations identify suspicious behavior and improve their defenses.
7. Test With Adversarial Inputs
Developers should test AI applications with malicious and unexpected inputs.
Security testing can reveal weaknesses before attackers discover them.
OWASP recommends adversarial testing and attack simulations as part of prompt injection defense.
A Simple Example of AI Security
Imagine a company builds an AI assistant for employees.
The assistant can:
Read company documents → Answer questions → Search internal systems
Now suppose an attacker adds malicious instructions to a document.
An employee asks the AI to summarize that document.
The AI processes both the document and its embedded instructions.
If the application has weak security boundaries, the malicious content could influence the AI’s behavior.
However, strong access controls can reduce the risk.
The application could:
- Treat the document as untrusted content.
- Limit the AI’s permissions.
- Validate the AI’s output.
- Require approval before sensitive actions.
- Monitor unusual requests.
Therefore, security should not depend entirely on the AI following its instructions perfectly.
Why RAG Applications Need Special Attention
Many modern AI applications use Retrieval-Augmented Generation (RAG).
RAG allows an AI system to retrieve information from external sources before generating an answer.
This approach can improve accuracy.
However, it also introduces additional input sources.
A retrieved document could contain malicious instructions. When the AI processes that document, the content may influence the model.
Consequently, developers should treat retrieved information as untrusted input.
OWASP notes that RAG and fine-tuning can improve AI applications but do not completely eliminate prompt injection vulnerabilities.
Prompt Injection Security Checklist
Developers can use this checklist when building AI applications:
- Separate trusted instructions from untrusted data.
- Treat external content as untrusted.
- Limit AI permissions.
- Follow least privilege.
- Validate AI-generated output.
- Require approval for high-risk actions.
- Monitor AI activity.
- Test with adversarial inputs.
- Review connected tools and APIs.
- Log important AI actions.
Together, these controls can reduce the potential impact of prompt injection.
Why Prompt Injection Matters
AI systems are becoming more capable.
They can now access documents, call APIs, search databases, and perform tasks automatically.
As a result, a prompt injection attack can have consequences beyond a strange chatbot response.
The more authority an AI application receives, the more important its security controls become.
Therefore, developers should design AI systems around the assumption that model input can be manipulated.
Final Thoughts
Prompt injection is a cybersecurity risk that targets how AI systems interpret instructions and data.
Attackers can use direct inputs or malicious content hidden inside external sources. In some cases, manipulation may expose information or influence connected tools.
However, organizations can reduce the risk.
Strong access controls, least privilege, output validation, human approval, monitoring, and adversarial testing can all help.
The most important principle is simple:
Do not give an AI system more authority than it needs.
AI should assist users, but security controls should remain outside the model wherever possible.
As AI becomes more connected to real-world systems, protecting those connections will become just as important as protecting the AI model itself.