search

52North / innovation-prize

1 stars 0 forks source link

Strategy for caching session data/Short-term-memory #29

Open simeonwetzel opened 1 month ago

simeonwetzel commented 1 month ago

A strategy is needed to decide where session data (state) is stored (server side / client side). This applies to API responses as well as chat history.

simeonwetzel commented 3 weeks ago

Implemention details:

  1. Database initialization: The server initializes a SQLite database on startup: https://github.com/52North/innovation-prize/blob/df7c0100f7b74753871daee1da6d5d0c9dc15ac7/search-app/server/app/server.py#L39

  2. Session creation: Each client must create a session by calling the create_session endpoint: curl -X POST http://localhost:8000/create_session https://github.com/52North/innovation-prize/blob/df7c0100f7b74753871daee1da6d5d0c9dc15ac7/search-app/server/app/server.py#L123

This endpoint generates a session_id, which is then stored in a session cookie: Image

  1. Session Management: When the langgraph state graph (chatbot) is initialized, the session_id is used to create a unique entry in the SQLite database. Each interaction with the graph (i.e., user input and AI response) results in a checkpoint being saved in the database. This checkpoint includes information such as:

    • Chat history
    • Search criteria
    • Spatial context
    • Search results This checkpoint functions as a form of short-term memory.

  2. Limitations and Benefits: Currently, checkpoints are not retained if the server is restarted. However, this approach allows different clients to maintain their own sessions and respective memories even when using a single SQLite database.