search

spring-projects / spring-ai

An Application Framework for AI Engineering
https://docs.spring.io/spring-ai/reference/1.0-SNAPSHOT/index.html
Apache License 2.0
2.55k stars 620 forks source link

Pass context information to Function Call Thread #991

Open dolukhanov opened 2 weeks ago

dolukhanov commented 2 weeks ago

Expected Behavior

When using Function Calling with OpenAI and other models - the function gets invoked in a new thread.

The function being called in my application requires context, so it can persist the response data to the correct user / session etc.

My application uses Spring Security - however, I anticipate there could be other requirements to pass context to this thread.

It would be ideal to be able to store the context before the thread is called and then retrieve the context e.g. using:

SecurityContext context = SecurityContextHolder.getContext();

and

SecurityContextHolder.setContext( context );

Perhaps you could register listeners / functions that can be called to pass this information.

Current Behavior

The calling thread does not have any context about the session / user that invoked the call to the AI model.

Context

The only workaround is to request that the model pass an identified to the function via the prompt.

miltonhit commented 1 week ago

I've opened a similar issue 3 weeks ago: https://github.com/spring-projects/spring-ai/issues/864

A possible workahound for it is passing a function "instance" in each interaction. Then you can pass some user/session in constructor.