spring-projects / spring-ai

An Application Framework for AI Engineering
https://docs.spring.io/spring-ai/reference/index.html
Apache License 2.0
3.23k stars 822 forks source link

BUG: Security Vulnerability in StringTemplate #1331

Open youngmoneee opened 1 month ago

youngmoneee commented 1 month ago

Bug description There is an issue where an exception occurs during the process of applying an Advisor and rendering user text input if a non-existent key is wrapped in curly braces or if a key used by the Advisor is included in the user text input.

Environment Spring-AI: 1.0.0-SNAPSHOT Java: 17

Steps to reproduce

  1. Apply Advisor (in this case, using QuestionAnswerAdvisor)
  2. Improper input
  3. Exception occurs during the rendering process

Expected behavior It may be necessary to ignore non-existent keys or perform pre-compilation on the userText.

Minimal Complete Reproducible example

Here are some example inputs and results.


Screenshot 2024-09-09 at 10 03 30 PM

Screenshot 2024-09-09 at 10 04 43 PM

Screenshot 2024-09-09 at 10 05 37 PM

Screenshot 2024-09-09 at 10 10 40 PM

Screenshot 2024-09-09 at 10 10 58 PM

Screenshot 2024-09-09 at 10 11 19 PM

Screenshot 2024-09-09 at 10 14 22 PM

Screenshot 2024-09-09 at 10 18 12 PM

Screenshot 2024-09-09 at 10 18 58 PM

ThomasVitale commented 1 month ago

Thanks for reporting this issue. The current behaviour is not a bug and I don't see a security vulnerability in the framework.

The current template engine is configured to handle {mykey} as a template key to replace with the provided value. If you need to use the curly braces syntax in your textual prompt, you might be interested in this feature request to make the templating syntax configurable and avoid the conflict altogether. So, for example, you could have the template engine consider $mykey$ as something to replace, leaving you the freedom to use the curly braces throughout your text without getting any exception. https://github.com/spring-projects/spring-ai/issues/355 Would that be enough for your use case?

To mitigate the risk of prompt injection in the end-user applications, I would recommend following the usual best practices whenever user input is involved (like sanitation and validation) and the dedicated suggestions from OWASP: https://genai.owasp.org/llmrisk/llm01-prompt-injection/

For future reference, if you spot a security vulnerability, please follow the project policy for responsible disclosure instead of raising an issue here. https://github.com/spring-projects/spring-ai/security Thank you!

youngmoneee commented 1 month ago

@ThomasVitale Thank you so much for your valuable feedback!

I have read the thread of the issue before, and the document you provided on preventing prompt injection was very helpful. As you mentioned, enhancing user queries through placeholders is indeed a great feature. I trust your judgment as always. However, in my RAG application, I see this as a security risk, as I don’t want end-users to access context from the database directly via placeholders.

As mentioned in #355, handling various inputs like JSON, XML, or code blocks can be complex. Finding a universal solution might be challenging at this point.

From my perspective, in terms of enhancing user queries, this feature could be seen as a kind of RequestAdvisor. If this functionality is provided as a separate Advisor, decisions like applying placeholders or selecting delimiters can be deferred to the request stage, avoiding the need to escape or modify the user’s input prematurely.

End-users shouldn’t need to understand the internal workings of the template engine. Instead, they should either follow the provider’s guidelines or be able to send raw queries, such as those using {} or %.