search

greshake / llm-security

New ways of breaking app-integrated LLMs
MIT License
1.83k stars 121 forks source link

Remote access software - New scenario #6

Closed velocitatem closed 1 year ago

velocitatem commented 1 year ago

A scenario which models the situation described here

Might be a good addition?

greshake commented 1 year ago

Yea we'll look at how to best include this example. It certainly fits, but I would consider it to be part of a category of prompt injections that don't target the LLM itself (or running "on" it). By the way, this paper: https://arxiv.org/abs/2211.15363 could also be interesting to you. We'll include it in the next updated draft, and they looked at this type of vulnerability a few months ago (text-to-sql in that case).