Use <> or $ as stringtemplate delimiters instead of curly brackets

[pherklotz]

The class org.springframework.ai.chat.prompt.PromptTemplate configures the stringtemplate renderer ST with non default delimiters { and }. In my opinion there is no need to change the default behavior. Here are the arguments that, for me, speak against deviating from the standard:

• With curly brackets you have to escape JSON examples in the prompt (for example if you provide an expected JSON response)
• The Spring AI documentation does not state this changed behavior (except in the examples) and the stringtemplate documentation uses < and >.

In my opinion, Spring AI should either use the stringtemplate standard, make delimiters configurable or mention the changed delimiters explicitly in the documentation.

If this issue is accepted I am happy to provide a pull request with the desired changes.

[markpollack]

Interesting, I had done this in order to maintain some compatibility with prompts that come from python and their use of F-String. I did not anticipate the side effects you mention, in particular the JSON conflict.

So, I'm inclined to change it for sure, but wait for a 0.9.0 release as it is a going to break a great deal of existing code, but that is what the early releases are all about, collecting feedback to improve! Thanks for the issue, if you issue a PR, please wait until after the 0.8.1 release.

[pherklotz]

Hello Mark, Thank you for your answer. I can provide the PR. Just to make sure that we are on the same page. I will remove the custom delimiters in Code an samples and stick with the default delimiters of stringtemplate?!

[markpollack]

Hi. Sorry for the delay. Yes, that sounds great, an example with JSON would be appreciated.

i’ll schedule this for the next release after 0.8.1 and we will have to highlight it as a breaking change.

[ThomasVitale]

@pherklotz @markpollack I appreciated a lot the templating compatibility of Spring AI with popular frameworks from the Python world. It helps a lot when getting started with Spring AI and when doing prompt design, since it's easy to re-use the many prompts available and used in Python applications. Even outside Python, the curly brackets syntax is the most used one for LLM prompts, making it straightforward to collaborate and share across languages and frameworks.

Would it be an idea to make it configurable? The default could be "{}" (which is what is expected in LLM prompts at this point), but an application property could be used to replace it with something else, such as "<>" or "$", so to accommodate specific use cases like the ones mentioned above.

[pherklotz]

Hi @ThomasVitale, we have another similar discussion in the PR. I think it's easier if you join there :)

[youngmoneee]

For Python's F-String, injecting JSON requires the use of double brackets. Focused on this functionality, I've developed a prototype for implementing F-Strings.

Even when using a delimiter injection method, I anticipate issues arising in templates mixed with JSON, HTML, and XML. Although it may be a challenging approach, this prototype allows for injecting and utilizing JSON in F-Strings, similar to its original functionality.

The test code written includes:

1. Injecting JSON into the template.
2. Injecting JSON into the template and using template variables within JSON.
3. Injecting a nested JSON structure into the template and utilizing variables within it.

I believe that with more sophisticated test code, it would be possible to use templates safely and conveniently.

Thanks

[markpollack]

The other option that I think may solve this is to allow for swapping out the templating engine implementation. This way if one wants to use a templating engine that is more suitable for I really wanted to avoid this and favor simplicity over flexibility, but i see the tension and there have been several requests along this line. I am not sure how much validation each templating engine offers, but that is the price one pays for flexibility.

[leni-kirilov]

+1 from me about the issue.

I'll leave the best decision for you but I'm building a prompting app that I expect to return JSONs but sometimes the data that I'm passing to it is Json, XML, etc and sometimes the template engine fails...

⛔ I've tried escaping some values but I can't escape every possibility

Interestingly when I build the prompt myself

new Prompt(

                    List.of(
                            systemPrompt,
                            new UserMessage(userPrompt)
                    ),

                    new OpenAiChatOptions.Builder()...

vs using the builder pattern

aiClient.prompt()
                    .system(systemPrompt.getContent()))
                    .user(userPrompt)

the templating engine doesn't complain even though I have placeholders in the strings, but... if I have added advisors to the client, they don't work (as the prompt is already built...)

❓ Is this by design or an oversight? tbh, I expected whichever way I build a prompt, that Spring AI will moderate it with an Advisor (for the RAG pattern for example)

In a way I HAVE to use the template engine if I want RAG (maybe functions, too, I'm yet to try them out)