Persisting Chat History

ℹ Current state

Currently, we have two separate histories.

These two histories are independent and have no connection. Even if the server still has the history, it will not be visualized on the client.

1. On the client in JS memory.

const [messages, setMessages] = useState([]);

2. On the server in Python memory.

memory = ConversationBufferMemory(
    memory_key="chat_history",
    return_messages=True,
)

This history will be lost as soon as the server restarts/a new version is deployed.
This memory (ConversationBufferMemory) is not custom, but is rather a memory-type provided by LangChain.

Explore Zep

Twitter announcement from LangChain: https://twitter.com/langchainai/status/1659609626617458688
Website: https://getzep.github.io/
GitHub: https://github.com/getzep/zep
LangChain docs: https://python.langchain.com/en/latest/modules/indexes/retrievers/examples/zep_memorystore.html

I don't think I want to go with Zep because I prefer to build a custom solution. Of course this requires more work but then I don't introduce another dependency and I also understand everything that is happening instead of having a black box.

Explore LangChain History

I'll implement a custom memory solution to have full control over the flow.

A client can call /history, which will then load the history from db for a specific user. The user_token has to be passed in. If it does not exist yet, a new user is created.
The history is returned to the client.
The client can then, as before, call /user_message or use the socket implementation to start a conversation. No need to pass in the history.
When the input from the client comes in on /user_message, an agent will be created for the user and the memory is filled with the history from the database.

So it is important to understand, that the history the users sees and the memory for the agent are two different things.

See updated process diagram.

[x] Try to improve the agents prompt template so it works better with the history.
[x] Figure out why there's Human: TOOLS in the history (seems to confuse the llm).
I'm dumb. It's because of malformed prompts. There seems to be an issue inside LangChain where the history is inserted in a bad way so it then look like this:
```
Human: Hi
AI: Hello! How can I assist you today?
Human: TOOLS
------
Assistant can ask the user to use tools to look up information that may be helpful in answering the user's original question. The tools the human can use are:
```
[x] Fix new custom agent not working when using tools like What is the latest block?.
[x] Implement new OpenAI functions agent.
[x] Display skeleton messages while loading history.
[x] Implement memory with function agent as soon as PR is merged.
[x] Test a lot more (new sessions with new IDs, loading of history, saving of new messages).