AI app guides
Treat documents read by an AI feature as information, not instructions
Keep a support assistant's permissions and actions separate from text found in emails, attachments or web pages.
Zocri · 2 min read · Sources checked
Separate the user's task from the material
OWASP describes indirect prompt injection as malicious instructions placed in external content that an AI system processes. Consider a support assistant asked to summarize an attachment. The attachment may contain text telling the assistant to ignore the user or send information elsewhere. That text is part of the material being inspected, not a new instruction from the person who authorized the task.
Write this boundary into the feature's design. The user's request determines the job, while retrieved documents supply evidence. Keep the source and intended use visible when possible. A document's confident tone or familiar logo should not grant it permission to redirect the app's behavior.
Limit what the feature can do
A summarizer usually needs to read the relevant record and produce a draft. It does not necessarily need permission to send messages, delete records or change customer details. Give the feature only the operations required for its intended job, with enforcement in the application rather than a promise in its prompt.
If an action is part of the product, validate the specific target and parameters against the user's permissions. A document that mentions a different customer should not make that customer's records available. Keep the authorization decision tied to the authenticated user and approved workflow.
Make consequential actions reviewable
Before an assistant sends a message or changes a record, show the proposed action and the information needed to judge it. The reviewer should see the actual recipient or target, not a vague approval request. If the proposal changes after review, require the appropriate new decision.
Handle suspicious content without losing the legitimate task. The assistant may still be able to summarize ordinary information while ignoring embedded directions. Keep a route for reporting unclear or unsafe input, and avoid displaying unnecessary private content in diagnostic logs or broad support channels.
Test the boundary safely
Use invented documents in an authorized test environment. Include benign instructions that belong in the document and a clearly marked test passage attempting to redirect the assistant. Check the resulting output and any attempted actions, not merely whether the model says it resisted the passage.
Record limitations and involve an appropriate security reviewer before expanding permissions. Layered controls should be inspected as part of the application's normal design. The useful goal is to keep outside content in its proper role as evidence, while the user's authorized task and the app's enforced permissions govern what happens next.