ValueError: Missing some input keys: {'context'} Langchain ConversationRetrievalChain

[UsamaHussain8]

Checked other resources

• [X] I added a very descriptive title to this issue.
• [x] I searched the LangChain documentation with the integrated search.
• [X] I am sure that this is a bug in LangChain rather than my code.
• [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The code for this problem is:

    embeddings = OpenAIEmbeddings(openai_api_key=configs.OPEN_API_KEY)
    #client = chromadb.Client(Settings(persist_directory="./trained_db"))
    client = chromadb.PersistentClient(path="./trained_db")
    collection = client.get_or_create_collection("PDF_Embeddings", embedding_function=embedding_functions.OpenAIEmbeddingFunction(api_key=config["OPENAI_API_KEY"], model_name=configs.EMBEDDINGS_MODEL))
    vectordb = Chroma(persist_directory="./trained_db", embedding_function=embeddings, collection_name = collection.name)
    prompt_template = f"""You are engaged in conversation with a human,
                          your responses will be generated using a comprehensive long document as a contextual reference. 
                          You can summarize long documents and also provide comprehensive answers, depending on what the user has asked.
                          You also take context in consideration and answer based on chat history.
                          Chat History: {{context}}

                          Question: {{question}}

                          Answer :
                        """

    PROMPT = PromptTemplate(template=prompt_template, input_variables=["context", "question"])

    model = configs.CHAT_MODEL
    streaming_llm = ChatOpenAI(openai_api_key=configs.OPEN_API_KEY, model = model, temperature = 0.1, streaming=True)

    # use the streaming LLM to create a question answering chain
    qa_chain = load_qa_chain(
        llm=streaming_llm,
        chain_type="stuff",
        prompt=PROMPT
    )
    question_generator_chain = LLMChain(llm=streaming_llm, prompt=PROMPT)

    qa_chain_with_history = ConversationalRetrievalChain(
    retriever = vectordb.as_retriever(search_kwargs={'k': 3}, search_type='mmr'),
    combine_docs_chain=qa_chain,
    question_generator=question_generator_chain
)
    response = qa_chain_with_history(
        {"question": query, "chat_history": user_specific_chat_memory.messages}
    )

    user_specific_chat_memory.add_user_message(response["question"])
    user_specific_chat_memory.add_ai_message(response["answer"])

    return {"code": "200", "answer": response["answer"]}

### Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 419, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\fastapi\applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\middleware\errors.py", line 186, in __call__
    raise exc
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\middleware\errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\middleware\exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\routing.py", line 758, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\routing.py", line 778, in app
    await route.handle(scope, receive, send)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\routing.py", line 299, in handle
    await self.app(scope, receive, send)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\routing.py", line 79, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\starlette\routing.py", line 74, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\fastapi\routing.py", line 299, in app
    raise e
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\fastapi\routing.py", line 294, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\fastapi\routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PDFChat\PDFChat\routers\chat.py", line 171, in pdf_chat
    response = qa_chain_with_history(
               ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\langchain_core\_api\deprecation.py", line 145, in warning_emitting_wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PDFChat\PDFChatEnv\Lib\site-packages\langchain\chains\base.py", line 378, in __call__
    return self.invoke(
           ^^^^^^^^^^^^
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\langchain\chains\base.py", line 163, in invoke
    raise e
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\langchain\chains\base.py", line 153, in invoke
    self._call(inputs, run_manager=run_manager)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\langchain\chains\conversational_retrieval\base.py", line 146, in _call
    new_question = self.question_generator.run(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\langchain_core\_api\deprecation.py", line 145, in warning_emitting_wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\langchain\chains\base.py", line 550, in run
    return self(kwargs, callbacks=callbacks, tags=tags, metadata=metadata)[
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:=\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\langchain_core\_api\deprecation.py", line 145, in warning_emitting_wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\langchain\chains\base.py", line 378, in __call__
    return self.invoke(
           ^^^^^^^^^^^^
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\langchain\chains\base.py", line 163, in invoke
    raise e
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\langchain\chains\base.py", line 151, in invoke
    self._validate_inputs(inputs)
  File "C:\PDFChat\PDFChat\PDFChatEnv\Lib\site-packages\langchain\chains\base.py", line 279, in _validate_inputs
    raise ValueError(f"Missing some input keys: {missing_keys}")
ValueError: Missing some input keys: {'context'}

### Description

I am trying to create a Chat application using LLM where user provides some pdf documents, the embeddings are generated and stored in a vector database (I am using chromadb for this purpose). So, when a user then submits a query, the model searches in the database for relevant documents and returns a response (essentially a RAG-based application). Also, I am storing the chat history in MongoDB. 

Now, when the user submits the first query, the response is generated fine. But when the user follows up, the application throws the error. 

I want the user to navigate in between different chats or at least the model to remember all the previous context (much like ChatGPT).

### System Info

Python Version:  3.11.6

> langchain_core: 0.1.32
> langchain: 0.1.12
> langchain_community: 0.0.28
> langsmith: 0.1.27
> langchain_mongodb: 0.1.1
> langchain_openai: 0.0.5
> langchain_text_splitters: 0.0.1
> langchainhub: 0.1.14

[honoured-1]

prompt_template = """You are engaged in conversation with a human,
                      your responses will be generated using a comprehensive long document as a contextual reference. 
                      You can summarize long documents and also provide comprehensive answers, depending on what the user has asked.
                      You also take context in consideration and answer based on chat history.
                      Chat History: {context}

                      Question: {question}

                      Answer :
                    """

From my understanding you don't need to write the prompt_template in a f string and write the context and question in single curly braces

[UsamaHussain8]

prompt_template = """You are engaged in conversation with a human,
                      your responses will be generated using a comprehensive long document as a contextual reference. 
                      You can summarize long documents and also provide comprehensive answers, depending on what the user has asked.
                      You also take context in consideration and answer based on chat history.
                      Chat History: {context}

                      Question: {question}

                      Answer :
                    """

From my understanding you don't need to write the prompt_template in a f string and write the context and question in single curly braces

Yeah I corrected that. However, it results in the same error.

[ya-smin20]

hello , did you find any solutions , i got the same issue

[awkyu]

any update here? im noticing that it works for the first .invoke() call, but fails with this same exception on the second call of .invoke() after updating/including a non-empty chat history. I notice the chain with an empty chat history goes from ConversationalRetrievalChain -> StuffDocumentsChain -> LLMChain vs with a non-empty chat history goes from ConversationalRetrievalChain -> LLMChain

UPDATE/WORKAROUND: This seems to be the case because when there is a chat history, it wants to rephrase the chat history and question into a condensed "standalone" question via the LLMChain. However, despite ConversationalRetrievalChain.from_llm seemingly setting the default question_generator to a chain with a prompt that doesn't have "context," it seems some default prompt with "context" as an input is used. So, the question_generator needs to be manually set after the from_llm function is used. Here is what I did:

CONDENSE_PROMPT = PromptTemplate.from_template(
    "Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question.\n\n"
    "Chat History:\n{chat_history}\nFollow Up Input: {question}\nStandalone question:"
)

question_generator = LLMChain(
    llm=llm,
    prompt=CONDENSE_PROMPT
)

doc_chain = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=retriever,
    condense_question_prompt=qa_prompt,
    # question_generator=question_generator, # setting this DOES NOT work either
    return_source_documents=True,
    rephrase_question=True,
    verbose=True,
)
doc_chain.question_generator = question_generator
doc_chain.ainvoke({'question': "some question there", 'chat_history': chat_history_list})