chat_history conflict when using ConversationalRetrievalChain.from_llm

[afif-malghani]

System Info

System Info:

langchain==0.0.325 Python 3.11.4 Flask==3.0.0 Flask-Cors==4.0.0 Flask-RESTful==0.3.10

Case 1:

dbqa = ConversationalRetrievalChain.from_llm(llm, vectordb.as_retriever(), memory=memory)
response = dbqa({"question": data["query"]})

Error: Missing some input keys: {'chat_history'}

Call Stack: Chain.__call__()

Chain.prepare_inputs(inputs) called from __call__() line 286 with inputs={'question': 'hey bot!'}

if self.memory is not None: in prepare_inputs() passed

external_context = self.memory.load_memory_variables(inputs) sets external_context to {'DocScanner.pdf': []}

self._validate_inputs(inputs) on line 443

missing_keys = set(self.input_keys).difference(inputs) results in missing_keys={'chat_history'}

Case 2:

dbqa = ConversationalRetrievalChain.from_llm(llm, vectordb.as_retriever(), memory=memory)
response = dbqa({"question": data["query"], "chat_history": chat_history})

Error: ValueError: One input key expected got ['chat_history', 'question']

Call Stack: Chain.__call__()

Chain.prepare_inputs(inputs) called from __call__() line 286 with inputs={'question': 'hey bot!'}

if self.memory is not None: in prepare_inputs() passed

external_context = self.memory.load_memory_variables(inputs) sets external_context to {'DocScanner.pdf': []}

self._validate_inputs(inputs) on line 443

missing_keys = set(self.input_keys).difference(inputs) results in missing_keys={}

Chiain.__call__ line 303 sets outputs={'answer': " Of course! I...elp today?'}

self.prep_outputs called on line 312.

inside prepare_outputs, if self.memory is not None: check passes

self.memory.save_context(inputs, outputs) calls to BaseChatMemory.save_context

input_str, output_str = self._get_input_output(inputs, outputs) calls to BaseChatMemory._get_input_output

if self.input_key is None: check passes

call to langchain.memory.utils.get_prompt_input_key

prompt_input_keys = list(set(inputs).difference(memory_variables + ["stop"])) returns ['chat_history', 'question']

if len(prompt_input_keys) != 1: check passes raising the error

Who can help?

Anyone

Information

• [X] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [ ] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [X] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

steps to reproduce:

1. clone repo
2. create a virtual environment with python3 -m venv env
3. activate the virtual environment with source env/bin/activate
4. install all packages with pip install -r requirements
5. run flask server with flask run --debug
6. upload a pdf at the home pate of the application displayed at http://localhost:5000/
7. In the Chat section enter the message you want to send to the bot.

Expected behavior

Response should be returned either with or without chat_history.

Currently it is not returned in either of the cases.

Edit: upgraded to langchain 0.0.325, issue persists. Edit 2: improved readability

[dosubot[bot]]

🤖

Hello @afif-malghani,

I'm a bot here to assist you with your issue while you're waiting for a human maintainer. Let's see if we can figure this out together.

From the error messages you've shared, it seems like the ConversationalRetrievalChain is expecting a chat_history key in the inputs. However, when you provide it, the format seems to be causing an issue.

Based on similar issues in the LangChain repository, it seems like the chat_history key is expected to be a list of either BaseMessage instances or tuples. If the format is not one of these, it raises a ValueError indicating an unsupported chat history format. You can find this in the _get_chat_history function in the base.py file of the conversational_retrieval directory (source).

In addition, there are a few solved issues that might be relevant to your case:

1. ConversationalRetrievalChain doesn't work with ConversationEntityMemory + SQLiteEntityStore: The issue was resolved by providing an extra key parameter to ConversationEntityMemory. The ConversationEntityMemory was returning the chat history through the key history, but the LLM chain was expecting chat_history.

2. Issue: Missing some input keys: {'question'} when using ConversationalRetrievalChain: The error "Missing some input keys: {'question'}" when using the ConversationalRetrievalChain in the LangChain framework can be resolved by modifying the line where the query is passed to the ConversationalRetrievalChain. Instead of using the key "query" to pass the query, the key "question" should be used as the ConversationalRetrievalChain is expecting this key.

3. ConversationalRetrievalChain doesn't work with memory: The issue of ConversationalRetrievalChain not utilizing memory for answering questions with references can be resolved by passing the chat_memory field in ConversationalBufferMemory before passing it to any chain.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Best, Dosu

Sources

• ConversationalRetrievalChain doesn't work with ConversationEntityMemory + SQLiteEntityStore
• Issue: Missing some input keys: {'question'} when using ConversationalRetrievalChain
• ConversationalRetrievalChain doesn't work with memory
• libs/langchain/langchain/chains/conversational_retrieval/base.py
• libs/langchain/tests/unit_tests/chains/test_conversation_retrieval.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[afif-malghani]

In response to Dosubot:

As per the documentation here when using qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature=0), vectorstore.as_retriever(), memory=memory) we do not need to pass history at all.

However when kwarg memory is not passed like so qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature=0), vectorstore.as_retriever()), then we need to pass memory.

So, since my implementation is according to the first case I described, I should not have to pass in the chat history at all.

[dosubot[bot]]

🤖

Based on the context provided, it seems like there's a mismatch between the keys expected by the ConversationalRetrievalChain.from_llm function and the keys provided by the ConversationEntityMemory.

In the issue #12266, a similar problem was encountered and it was resolved by providing an extra key parameter to ConversationEntityMemory. The ConversationEntityMemory was returning the chat history through the key history, but the LLM chain was expecting chat_history. The code was modified as follows:

memory = ConversationEntityMemory(
            llm=llm,
            entity_store=entity_store,
            chat_history_key='chat_history', # This one was provided to make it work
            return_messages=True
        )

This modification made the keys match and resolved the issue.

However, if you're not using a chat history in your implementation, you can try setting the memory_key to None in the ConversationBufferMemory function. This should prevent the ConversationalRetrievalChain from looking for a chat history. Here's how you can modify the ConversationBufferMemory function call:

memory = ConversationBufferMemory(
    k=1, output_key="answer", memory_key=None, return_messages=True
)

Please try these modifications and see if they resolve your issue. If the problem persists, please provide more details about your implementation, especially how you're initializing and using the ConversationEntityMemory and ConversationalRetrievalChain.from_llm functions.

Sources

• ConversationalRetrievalChain doesn't work with ConversationEntityMemory + SQLiteEntityStore
• libs/langchain/tests/unit_tests/chains/test_conversation_retrieval.py
• libs/langchain/langchain/chains/conversational_retrieval/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[afif-malghani]

In response to the previous suggestion:

setting memory_key=None results in the following error:

none is not an allowed value (type=type_error.none.not_allowed)

[afif-malghani]

Issue resolved by using tuple as history instead of list.

request: Please update documentation here accordingly

[afif-malghani]

Facing the same problem again

[kotowicz]

I'm facing the same problem. Starting with a fresh memory this code works

memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True, output_key='answer')
qa_chain = ConversationalRetrievalChain.from_llm(llm, memory=memory, retriever=vectorstore.as_retriever(), return_source_documents=True, get_chat_history=lambda h : h, verbose=True)

chat_history_tuples = []
for message in chat_history:
    chat_history_tuples.append((message[0], message[1]))
result = qa_chain({"question": query, "chat_history": chat_history_tuples})

Now let's restore the messages and build the qa_chain

retrieved_messages = messages_from_dict(retrieve_from_db)
retrieved_chat_history = ChatMessageHistory(messages=retrieved_messages)
retrieved_memory = ConversationBufferMemory(chat_memory=retrieved_chat_history)

qa_chain = ConversationalRetrievalChain.from_llm(llm, memory=retrieved_memory, retriever=vectorstore.as_retriever(), return_source_documents=True, get_chat_history=lambda h : h, verbose=True)

The restored chat_memory looks good

chat_memory=ChatMessageHistory(messages=[HumanMessage(content='what is the document about?'), AIMessage(content='The document is ab.... success.'), HumanMessage(content='and what about the financial sphere?'), AIMessage(content='The document mentions that the financial sphere ....')])

The same code as above result = qa_chain({"question": query, "chat_history": chat_history_tuples}) throws the error:

Error: One input key expected got ['question', 'chat_history']
Traceback (most recent call last):
  File "/Users/pc_interface_gradio.py", line 28, in user
    result = self.qa_function({"question": query, "chat_history": chat_history_tuples})
  File "/Users/lib/python3.10/site-packages/langchain/chains/base.py", line 312, in __call__
    final_outputs: Dict[str, Any] = self.prep_outputs(
  File "/Users/python3.10/site-packages/langchain/chains/base.py", line 408, in prep_outputs
    self.memory.save_context(inputs, outputs)
  File "/Users/lib/python3.10/site-packages/langchain/memory/chat_memory.py", line 35, in save_context
    input_str, output_str = self._get_input_output(inputs, outputs)
  File "/Users/lib/python3.10/site-packages/langchain/memory/chat_memory.py", line 22, in _get_input_output
    prompt_input_key = get_prompt_input_key(inputs, self.memory_variables)
  File "/Users/lib/python3.10/site-packages/langchain/memory/utils.py", line 19, in get_prompt_input_key
    raise ValueError(f"One input key expected got {prompt_input_keys}")
ValueError: One input key expected got ['question', 'chat_history']

Now, when I change the line to result = qa_chain({"question": query}) the error becomes

Error: Missing some input keys: {'chat_history'}
Traceback (most recent call last):
  File "/Users/pc_interface_gradio.py", line 28, in user
    result = self.qa_function({"question": query, "chat_history": chat_history_tuples})
  File "/Users/lib/python3.10/site-packages/langchain/chains/base.py", line 286, in __call__
    inputs = self.prep_inputs(inputs)
  File "/Users/lib/python3.10/site-packages/langchain/chains/base.py", line 443, in prep_inputs
    self._validate_inputs(inputs)
  File "/Users/lib/python3.10/site-packages/langchain/chains/base.py", line 195, in _validate_inputs
    raise ValueError(f"Missing some input keys: {missing_keys}")
ValueError: Missing some input keys: {'chat_history'}

[afif-malghani]

@kotowicz have you tried changing memory_key to a variable. For me that is where the error is being triggered from. If I keep it chat_history it works fine. Can you try memory_key="chat_history" in retrieval? maybe it will work. However for me I need multiple chat histories therefore I need the memory_key to be dynamic.

[kotowicz]

@afif-malghani thanks for your feedback the following does indeed work!

retrieved_memory = ConversationBufferMemory(memory_key="chat_history", chat_memory=retrieved_chat_history, return_messages=True, output_key='answer')

Which is identical (except for the chat_memory part) to how I initiated the memory initially:

initial_memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True, output_key='answer')

Thanks!

[afif-malghani]

Update: passing memory_key="any_other_string" fails, the only memory_key acceptable is chat_history.

[dosubot[bot]]

Hi, @afif-malghani,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, the issue involves a conflict with the chat_history input key when using ConversationalRetrievalChain.from_llm. There have been attempts to resolve it by modifying the ConversationEntityMemory and ConversationBufferMemory functions, and using a tuple as history instead of a list. Discussions have also taken place regarding the acceptability of different values for the memory_key parameter.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!