search

redis / redis-vl-python

Redis Vector Library (RedisVL) interfaces with Redis' vector database for realtime semantic search, RAG, and recommendation systems.
https://www.redisvl.com/
MIT License
176 stars 25 forks source link

Add Session Management Feat. #170

Open ajac-zero opened 1 week ago

ajac-zero commented 1 week ago

Has progress begun on llm session management? It would be an awesome feature.

Currently, our apps pull past conversation messages straight from our db upon each request, identified by a conversation id.

However, saving on-going conversations in-cache would be much faster and off-load some work from the db.

Is there an idea on how this would be implemented? Or should I pr something from scratch?

justin-cechmanek commented 1 week ago

It's under development and should be available soon.

tylerhutcherson commented 1 week ago

@ajac-zero work is available on this branch/PR https://github.com/redis/redis-vl-python/pull/141 Your feedback is welcomed.

ajac-zero commented 1 week ago

Awesome! The semantic session manager is really interesting.

I noticed each exchange is expected to be a pair of strings, would this limit the use of tool messages? Due to most providers returning tool/function calls as dicts. I suppose the simplest solution would be to stringify the response dicts before storing them, although this might require additional parsing. Alternatively using json mode would work fine already if available.

Also, there are situations where a single exchange could involve more than 2 messages. Like when a llm uses tools before generating a response.

user query message -> assistant tool call message -> tool result message -> assistant response message

For cases like these it could be useful to provide the option to store messages individually to allow greater flexibility.

justin-cechmanek commented 1 day ago

Hi @ajac-zero. The session managers do have a convenience store method to add a prompt and response texts in one call. This is meant for typical prompt->response->prompt->... chats. You can also call add_message to add messages individually in the form of dicts and these include tool call messages. Can you let us know if that meets your needs?