Conversation memory example - previous messages not remembered

[joshpxyne]

System Info

LangChain version 0.0.330 Chroma version 0.4.15

This may either be a true bug or just documentation issue, but I implemented the simplest possible version of a ConversationalRetrievalChain nearly directly from the documentation, and the model doesn't remember previous messages.

from langchain.llms import OpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Chroma

embeddings = OpenAIEmbeddings()
vectorstore = Chroma(embedding_function=embeddings)

from langchain.memory import ConversationBufferMemory
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature=0,model_name="gpt-4-32k"), vectorstore.as_retriever(), memory=memory)

query = "My name's Bob. How are you?"
result = qa({"question": query})

print(result)

query = "What's my name?"
result = qa({"question": query})

print("NEW MESSAGE:",result)

The output shows that the model has no memory:

{'question': "My name's bob. How are you?", 'chat_history': [HumanMessage(content="My name's bob. How are you?", additional_kwargs={}, example=False), AIMessage(content="I'm doing well, thank you. How can I assist you today, Bob?", additional_kwargs={}, example=False)], 'answer': "I'm doing well, thank you. How can I assist you today, Bob?"}
NEW MESSAGE: {'question': "What's my name?", 'chat_history': [HumanMessage(content="My name's bob. How are you?", additional_kwargs={}, example=False), AIMessage(content="I'm doing well, thank you. How can I assist you today, Bob?", additional_kwargs={}, example=False), HumanMessage(content="What's my name?", additional_kwargs={}, example=False), AIMessage(content='The text does not provide the name of the person who is speaking.', additional_kwargs={}, example=False)], 'answer': 'The text does not provide the name of the person who is speaking.'}

Apologies if I missed something dumb but it seemed pretty cut and dry so felt like I at least had to post for posterity's sake.

Who can help?

No response

Information

• [ ] The official example notebooks/scripts
• [X] My own modified scripts

Related Components

• [X] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [X] Vector Stores / Retrievers
• [X] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [X] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

1. run this code:

from langchain.llms import OpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Chroma

embeddings = OpenAIEmbeddings()
vectorstore = Chroma(embedding_function=embeddings)

from langchain.memory import ConversationBufferMemory
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature=0,model_name="gpt-4-32k"), vectorstore.as_retriever(), memory=memory)

query = "My name's Bob. How are you?"
result = qa({"question": query})

print(result)

query = "What's my name?"
result = qa({"question": query})

print("NEW MESSAGE:",result)

Expected behavior

I'd expect the model to remember "my name".

[dosubot[bot]]

🤖

Hello @joshpxyne,

I'm Dosu, a friendly bot here to assist you with LangChain! While we're waiting for a human maintainer, I can help you troubleshoot bugs, answer questions, and even guide you on how to be a contributor. I'm currently reviewing your issue and will provide a full response shortly.

[dosubot[bot]]

🤖

Thank you for bringing this to our attention. The issue you're experiencing seems to be related to how the ConversationalRetrievalChain handles memory and conversation history. In the current implementation, the chat history is not being updated after each conversation turn, which is why the model doesn't remember previous messages.

In the _call and _acall methods of the BaseConversationalRetrievalChain class, the chat history is retrieved from the inputs and converted into a string format using the get_chat_history function. However, after the conversation turn is processed, the chat history is not updated with the new messages. Here is the relevant code:

def _call(
    self,
    inputs: Dict[str, Any],
    run_manager: Optional[CallbackManagerForChainRun] = None,
) -> Dict[str, Any]:
    _run_manager = run_manager or CallbackManagerForChainRun.get_noop_manager()
    question = inputs["question"]
    get_chat_history = self.get_chat_history or _get_chat_history
    chat_history_str = get_chat_history(inputs["chat_history"])
    ...
    new_inputs["chat_history"] = chat_history_str
    answer = self.combine_docs_chain.run(
        input_documents=docs, callbacks=_run_manager.get_child(), **new_inputs
    )
    ...
    return output

In the above code, new_inputs["chat_history"] is set to chat_history_str which is the string representation of the chat history at the start of the conversation turn. This means that the chat history does not get updated with the new messages from the current conversation turn.

To fix this issue, you would need to update the chat history with the new messages after each conversation turn. This could be done by appending the new messages to the chat history list before converting it into a string format.

I also found some similar issues in the LangChain repository that might be helpful:

1. Working of ConversationBufferMemory in the context of document based question answering
2. The memory did not record the full input message, it only record the input parameter values
3. ConversationalRetrievalChain doesn't work with memory
4. How to add conversational memory to pandas toolkit agent?

I hope this helps! If you have any other questions or need further clarification, please let me know.

Sources

• Working of ConversationBufferMemory in the context of document based question answering
• The memory did not record the full input message, it only record the input parameter values
• ConversationalRetrievalChain doesn't work with memory
• How to add conversational memory to pandas toolkit agent?
• libs/langchain/langchain/chains/conversational_retrieval/base.py
• libs/langchain/langchain/memory/chat_message_histories/elasticsearch.py
• libs/langchain/tests/unit_tests/chains/test_conversation_retrieval.py
• libs/langchain/langchain/memory/zep_memory.py
• libs/langchain/langchain/memory/entity.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[joshpxyne]

@dosu-bot Based on the docs, I shouldn't need to pass the conversation history in explicitly. See the below copy/paste of the docs; the first example implements memory without passing it in explicitly, and the second example implements it with it. Both should be possible.

Remembering chat history The ConversationalRetrievalQA chain builds on RetrievalQAChain to provide a chat history component.

It first combines the chat history (either explicitly passed in or retrieved from the provided memory) and the question into a standalone question, then looks up relevant documents from the retriever, and finally passes those documents and the question to a question-answering chain to return a response.

To create one, you will need a retriever. In the below example, we will create one from a vector store, which can be created from embeddings.

from langchain.embeddings.openai import OpenAIEmbeddings from langchain.vectorstores import Chroma from langchain.text_splitter import CharacterTextSplitter from langchain.llms import OpenAI from langchain.chains import ConversationalRetrievalChain

Load in documents. You can replace this with a loader for whatever type of data you want

from langchain.document_loaders import TextLoader loader = TextLoader("../../state_of_the_union.txt") documents = loader.load()

If you had multiple loaders that you wanted to combine, you do something like:

loaders = [....]

docs = []

for loader in loaders:

docs.extend(loader.load())

We now split the documents, create embeddings for them, and put them in a vectorstore. This allows us to do semantic search over them.

text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0) documents = text_splitter.split_documents(documents)

embeddings = OpenAIEmbeddings() vectorstore = Chroma.from_documents(documents, embeddings)

Using embedded DuckDB without persistence: data will be transient

We can now create a memory object, which is necessary to track the inputs/outputs and hold a conversation.

from langchain.memory import ConversationBufferMemory memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

We now initialize the ConversationalRetrievalChain

qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature=0), vectorstore.as_retriever(), memory=memory)

query = "What did the president say about Ketanji Brown Jackson" result = qa({"question": query})

result["answer"]

" The president said that Ketanji Brown Jackson is one of the nation's top legal minds, a former top litigator in private practice, a former federal public defender, and from a family of public school educators and police officers. He also said that she is a consensus builder and has received a broad range of support from the Fraternal Order of Police to former judges appointed by Democrats and Republicans."

query = "Did he mention who she succeeded" result = qa({"question": query})

result['answer']

' Ketanji Brown Jackson succeeded Justice Stephen Breyer on the United States Supreme Court.'

Pass in chat history In the above example, we used a Memory object to track chat history. We can also just pass it in explicitly. In order to do this, we need to initialize a chain without any memory object.

qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature=0), vectorstore.as_retriever())

Here's an example of asking a question with no chat history

chat_history = [] query = "What did the president say about Ketanji Brown Jackson" result = qa({"question": query, "chat_history": chat_history})

result["answer"]

" The president said that Ketanji Brown Jackson is one of the nation's top legal minds, a former top litigator in private practice, a former federal public defender, and from a family of public school educators and police officers. He also said that she is a consensus builder and has received a broad range of support from the Fraternal Order of Police to former judges appointed by Democrats and Republicans."

Here's an example of asking a question with some chat history

chat_history = [(query, result["answer"])] query = "Did he mention who she succeeded" result = qa({"question": query, "chat_history": chat_history})

result['answer']

' Ketanji Brown Jackson succeeded Justice Stephen Breyer on the United States Supreme Court.'

[dosubot[bot]]

🤖

Based on the context provided, it appears that the ConversationalRetrievalChain in the LangChain Python framework is not remembering previous messages in the chat history because the chat history is not being stored or updated after each conversation turn. The chat history is only used within the scope of a single call to the chain, and is not preserved for future calls.

To resolve this issue, you can use any of the various chat message history classes provided in the LangChain framework, such as ElasticsearchChatMessageHistory, CassandraChatMessageHistory, CosmosDBChatMessageHistory, DynamoDBChatMessageHistory, and many more. These classes store the chat history in their respective databases, which allows the history to be preserved across multiple calls to the chain.

Here is an example of how you can use the ElasticsearchChatMessageHistory class:

from langchain.memory.chat_message_histories.elasticsearch import ElasticsearchChatMessageHistory
from langchain.schema.messages import TextMessage

# Initialize the chat message history
chat_history = ElasticsearchChatMessageHistory(index="chat_history", session_id="session1", es_url="http://localhost:9200")

# Assume `chain` is an instance of ConversationalRetrievalChain
while True:
    question = input("Enter your question: ")  # Get a question from the user
    question_message = TextMessage(role="human", content=question)  # Create a message object for the question
    chat_history.add_message(question_message)  # Add the user's question to the chat history

    # Run the chain with the current chat history
    result = chain.run({"question": question, "chat_history": chat_history.messages})

    print("AI's answer:", result["answer"])  # Print the AI's answer
    answer_message = TextMessage(role="ai", content=result["answer"])  # Create a message object for the answer
    chat_history.add_message(answer_message)  # Add the AI's answer to the chat history

In this example, chat_history is an instance of ElasticsearchChatMessageHistory. The add_message method is used to add new messages to the chat history, and the messages property is used to retrieve the current chat history. The chat history is stored in an Elasticsearch index, which allows it to be preserved across multiple calls to the chain. This way, the chain has access to the entire chat history for each new question.

You can choose the chat message history class that best suits your needs based on the database you prefer to use. For more information, you can refer to the following sources:

• libs/langchain/langchain/chains/conversational_retrieval/base.py
• libs/langchain/langchain/memory/chat_message_histories/elasticsearch.py
• libs/langchain/langchain/memory/chat_message_histories/init.py

I hope this helps! If you have any further questions, feel free to ask.

Sources

• ConversationalRetrievalChain doesn't work with memory
• ConversationRetrievalChain with memory
• Chatbot memory integration
• How to add conversational memory to pandas toolkit agent?
• libs/langchain/langchain/chains/conversational_retrieval/base.py
• libs/langchain/langchain/memory/chat_message_histories/elasticsearch.py
• libs/langchain/langchain/memory/chat_message_histories/init.py
• libs/langchain/tests/unit_tests/chains/test_conversation_retrieval.py
• libs/langchain/langchain/memory/zep_memory.py
• libs/langchain/langchain/memory/init.py
• libs/langchain/langchain/memory/entity.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[dosubot[bot]]

Hi, @joshpxyne,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, you reported an issue with the ConversationalRetrievalChain model not remembering previous messages as expected. Dosu-bot provided a detailed analysis and suggested using chat message history classes provided in the LangChain framework, specifically recommending the ElasticsearchChatMessageHistory class. You responded with a code snippet from the documentation, and multiple users reacted to Dosu-bot's response.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

[Danouchka]

memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True, output_key="answer")