Adding example attack scenario for ReAct agents

[kyuz0]

I propose to add a prompt injection attack scenario specific to organizations that are looking into implementing autonomous ReAct based agents to expose to customers, such as those that can be be built with Langchain. In these agents, it is possible to leverage prompt injection to inject Thoughts, Actions and Observations that alter the behaviour of the agent, but not via a direct jailbreak (i.e.: deviation from the system prompt) - instead they do so by altering the external reality of the agent.

Reference and attack example: https://labs.withsecure.com/publications/llm-agent-prompt-injection

Probably better understood in action in this short 30-sec video: https://www.linkedin.com/posts/withsecure_prompt-injection-for-react-llm-agents-ugcPost-7125756992341618688-ZFQ5

It's debatable though where such a scenario would fit: it's an injection attack, but not a direct jail break. The "LLM08: Excessive Agency" vulnerability definitely has an overlap, so does the the "LLM07: Insecure Plugin Design" to an extent. In this scenario mentioned by the article the agent should not be able to issue a refund if the original order doesn't exist or if information doesn't match the records.

[GangGreenTemperTatum]

Hey @kyuz0

Thanks for your suggestion and feedback. However, please submit a GitHub issue against this project with LLM-01 label for prompt injection as the work is being triaged to the entry leads and the easiest way for us to manage triage and a single source of truth in the vulnerability entries.

Thanks