Error in RootListenersTracer.on_chain_end callback: KeyError('answer') while streaming a RAG Chain Output

[Alptimus]

Checked other resources

• [x] I added a very descriptive title to this issue.
• [X] I searched the LangChain documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain rather than my code.
• [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

I have the following code for building a RAG Chatbot (using this example):

from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain.chains import create_retrieval_chain
from langchain.chains.history_aware_retriever import create_history_aware_retriever
from langchain.chains.combine_documents import create_stuff_documents_chain

vectordb = FAISS.load_local(persist_directory, embedding, index_name, allow_dangerous_deserialization=True)
retriever=vectordb.as_retriever()

llm = ChatOpenAI()

....
prompt={.....}
....

question_answer_chain = create_stuff_documents_chain(llm, prompt, output_parser=parser)
rag_chain = create_retrieval_chain(history_aware_retriever, question_answer_chain)

conversational_rag_chain = RunnableWithMessageHistory(
    rag_chain,
    get_session_history,
    input_messages_key="input",
    history_messages_key="chat_history",
    output_messages_key="answer",
)

while True:
    query = input("Ask a question: ")

    for chunk in conversational_rag_chain.stream(
        {"input": query,},
        config={
            "configurable": {
                "session_id": "demo_1"
            }
        }
    ):
        if answer_chunk := chunk.get("answer"):
            print(f"{answer_chunk}", end="", flush=True)

    print()

Error Message and Stack Trace (if applicable)

Ask a question: How many colors are in rainbow?
Error in RootListenersTracer.on_chain_end callback: KeyError('answer')
Error in callback coroutine: KeyError('answer')
A rainbow typically has seven colors, which are: Red, Orange, Yellow, Green, Blue, Indigo, Violet.</s>
Ask a question: 

Description

Hi,

I am trying to get the answer as stream, the problem is whenever the conversational_rag_chain.stream() is initiating, using an input it is giving the following errors:
Error in RootListenersTracer.on_chain_end callback: KeyError('answer')
Error in callback coroutine: KeyError('answer')
and then the output is printing as intended.

My question is, how can I solve it? I have entered output_messages_key="answer" in the conversational_rag_chain already, so am I doing something wrong or is a bug?

Any little discussion or help is welcome. Thanks in advance.

System Info

System Information

OS: Linux OS Version: #39-Ubuntu SMP PREEMPT_DYNAMIC Fri Jul 5 21:49:14 UTC 2024 Python Version: 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0]

Package Information

langchain_core: 0.2.23 langchain: 0.2.11 langchain_community: 0.2.10 langsmith: 0.1.93 langchain_chroma: 0.1.2 langchain_openai: 0.1.17 langchain_text_splitters: 0.2.2

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langgraph langserve

[ShaiDiamant]

Having the same issue while upgraded to the last version Important note - removing the "output_messages_key="answer", results in a different error + breaks the logic of the chat history, so don't attempt that

[ssultan19]

having the same issue. Below are my pip packages versions: langchain==0.2.3 langchain-cli==0.0.24 langchain-community==0.2.4 langchain-core==0.2.23 langchain-elasticsearch==0.2.2 langchain-mongodb==0.1.7 langchain-openai==0.1.8 langchain-pinecone==0.1.1 langchain-text-splitters==0.2.1 langserve==0.2.1 langsmith==0.1.75

[gustavo-sacchi]

I 'm having the same issue. ''' Error in RootListenersTracer.on_chain_end callback: KeyError('message_history') Error in callback coroutine: KeyError('message_history') Error in RootListenersTracer.on_chain_end callback: KeyError('message_history') Error in callback coroutine: KeyError('message_history') ''' Pip packages versions: langchain==0.2.11 langchain-anthropic==0.1.17 langchain-community==0.2.10 langchain-core==0.2.23 langchain-experimental==0.0.63 langchain-google-genai==1.0.7 langchain-groq==0.1.6 langchain-mistralai==0.1.9 langchain-openai==0.1.13 langchain-postgres==0.0.9 langchain-qdrant==0.1.3 langchain-text-splitters==0.2.2 langcodes==3.4.0 langfuse==2.36.2

[sherlock-tez]

Me too lol

[madonzy]

I have similar, but with another key:

Error in RootListenersTracer.on_chain_end callback: KeyError('message_history')
Error in callback coroutine: KeyError('message_history')
Error in RootListenersTracer.on_chain_end callback: KeyError('message_history')
Error in callback coroutine: KeyError('message_history')

[agvinaykr]

getting same error: Error in RootListenersTracer.on_chain_end callback: KeyError('output')

[JeongYunLee]

me too..

[kangcp]

Update to the latest version can not be used directly, the experience is very poor, I do not know how langchain developers do the test

[DonaghBr]

I have the same issue but only yesterday. Worked perfectly last week, but I did pull again from langchain: Error in RootListenersTracer.on_chain_end callback: KeyError('message_history') Error in callback coroutine: KeyError('message_history') Error in RootListenersTracer.on_chain_end callback: KeyError('message_history') Error in callback coroutine: KeyError('message_history')

langchain==0.2.11

[Alptimus]

I have the same issue but only yesterday. Worked perfectly last week, but I did pull again from langchain: Error in RootListenersTracer.on_chain_end callback: KeyError('message_history') Error in callback coroutine: KeyError('message_history') Error in RootListenersTracer.on_chain_end callback: KeyError('message_history') Error in callback coroutine: KeyError('message_history')

langchain==0.2.11

Can you tell which langchain version you were using till last week before pulling 0.2.11?

[pjurczen]

Same issue after upgrade to 0.2.11 (still present in 0.2.12): WARNING:langchain_core.callbacks.manager:Error in AsyncRootListenersTracer.on_chain_end callback: KeyError('output_text') Error in AsyncRootListenersTracer.on_chain_end callback: KeyError('output_text')

[edwardgonen]

Have the same problem. But I have another VENV that does not have it. Here are the versions that seem to work ok: langchain 0.2.7 langchain-community 0.2.6 langchain-core 0.2.12

I'm not sure these are the exact files that cause the problem. But intuitively sounds like them

[vaibhav-s-dabhade]

I have same issue:

conversational_rag_chain = RunnableWithMessageHistory( retrieval_chain, self.get_session_history, input_messages_key="input", history_messages_key="chat_history", output_messages_key="answer", )

conversational_rag_chain.invoke( {"input": "List all the names"}, config={"configurable": {"session_id": "vaibhav-123"}}, )["answer"]

above with call to invoke works well but as shown below call to astream gives error (getting output is as expected)

async for chunk in conversational_rag_chain.astream( {"input": "List all the names"}, config={ "configurable": {"session_id": "vaibhav-123"} }, ):

Error in AsyncRootListenersTracer.on_chain_end callback: KeyError('answer') Error in RootListenersTracer.on_chain_end callback: KeyError('answer')

[edwardgonen]

BTW - despite the error, seems like the functionality is not affected. Am I right?

[lopozz]

I don't see the error with this versioning: langchain-community==0.2.9 langchain==0.2.9 langchain-core==0.2.22 langchain-openai==0.1.10

[XAGU]

have same issue

[weavetolga]

BTW - despite the error, seems like the functionality is not affected. Am I right?

Seems like it, Im experiencing the same behavior with v0.2.11

[arishokri]

Same here!

[crfgxr]

I don't see the error with this versioning: langchain-community==0.2.9 langchain==0.2.9 langchain-core==0.2.22 langchain-openai==0.1.10

same here... above suggested package combinations works without getting the error.

[vishalnagda1]

I have observed that the issue starts occurring from langchain-core version 0.2.23. To resolve this, I downgraded langchain-core to version 0.2.22, and it is now working fine.

Additionally, please note that the versions of langchain = "0.2.10" and langchain-community = "0.2.9" cannot be upgraded beyond these versions, as they are the last versions supported up to langchain-core 0.2.22.

I hope this helps in resolving the issue for others experiencing similar problems.

[maximececcaldi]

Thanks @vishalnagda1 , I dowgraded as you said and it works now perfectly. Thank you so much for the help, it is well appreciated !

[Simon-Stone]

Spent around two hours today chasing this down until I finally found this issue. Thank you so much!

[MatheusRDG]

@vishalnagda1 Thx, solved here!

[kunwarmahen]

I am on version

Name: langchain-core Version: 0.2.29 Summary: Building applications with LLMs through composability

Please make sure to import from correct location. I found the sample code pointed to different location e.g. correcting to following fixed mine

from langchain.chains.retrieval import create_retrieval_chain from langchain.chains.history_aware_retriever import create_history_aware_retriever from langchain.chains.combine_documents import create_stuff_documents_chain

[omgbbq]

Same issue, when I update langchain_openai.

  - Updating langchain-core (0.2.19 -> 0.2.29)
  - Updating langchain-openai (0.1.16 -> 0.1.20)

Error in AsyncRootListenersTracer.on_chain_end callback: ValueError()
Error in RootListenersTracer.on_chain_end callback: ValueError()

[dishuwang]

have the same issue when upgrade langchain-core to 0.2.29, solved by downgrade langchain-core to 0.2.28

[annv-ltv]

Thank @dishuwang, is solved.

[evisong]

After upgrading to newer versions, the issue gets solved for me.

  - Updating langchain (0.2.12 -> 0.2.14)
  - Updating langchain-aws (0.1.15 -> 0.1.16)
  - Updating langchain-openai (0.1.20 -> 0.1.21)

Thanks for efforts.

[edwardgonen]

I confirm that pip install -U langchain solves the problem. The following packages updated: langchain-0.2.14 langchain-core-0.2.32

[vaibhav-s-dabhade]

I have same issue:

conversational_rag_chain = RunnableWithMessageHistory( retrieval_chain, self.get_session_history, input_messages_key="input", history_messages_key="chat_history", output_messages_key="answer", )

conversational_rag_chain.invoke( {"input": "List all the names"}, config={"configurable": {"session_id": "vaibhav-123"}}, )["answer"]

above with call to invoke works well but as shown below call to astream gives error (getting output is as expected)

async for chunk in conversational_rag_chain.astream( {"input": "List all the names"}, config={ "configurable": {"session_id": "vaibhav-123"} }, ):

Error in AsyncRootListenersTracer.on_chain_end callback: KeyError('answer') Error in RootListenersTracer.on_chain_end callback: KeyError('answer')

I have same issue:

conversational_rag_chain = RunnableWithMessageHistory( retrieval_chain, self.get_session_history, input_messages_key="input", history_messages_key="chat_history", output_messages_key="answer", )

conversational_rag_chain.invoke( {"input": "List all the names"}, config={"configurable": {"session_id": "vaibhav-123"}}, )["answer"]

above with call to invoke works well but as shown below call to astream gives error (getting output is as expected)

async for chunk in conversational_rag_chain.astream( {"input": "List all the names"}, config={ "configurable": {"session_id": "vaibhav-123"} }, ):

Error in AsyncRootListenersTracer.on_chain_end callback: KeyError('answer') Error in RootListenersTracer.on_chain_end callback: KeyError('answer')

Error is no more there - upgraded to latest versions as below: langchain : 0.2.14 langchain-core : 0.2.32

[vaibhav-s-dabhade]

Checked other resources

• [x] I added a very descriptive title to this issue.
• [x] I searched the LangChain documentation with the integrated search.
• [x] I used the GitHub search to find a similar question and didn't find it.
• [x] I am sure that this is a bug in LangChain rather than my code.
• [x] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

I have the following code for building a RAG Chatbot (using this example):

from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain.chains import create_retrieval_chain
from langchain.chains.history_aware_retriever import create_history_aware_retriever
from langchain.chains.combine_documents import create_stuff_documents_chain

vectordb = FAISS.load_local(persist_directory, embedding, index_name, allow_dangerous_deserialization=True)
retriever=vectordb.as_retriever()

llm = ChatOpenAI()

....
prompt={.....}
....

question_answer_chain = create_stuff_documents_chain(llm, prompt, output_parser=parser)
rag_chain = create_retrieval_chain(history_aware_retriever, question_answer_chain)

conversational_rag_chain = RunnableWithMessageHistory(
    rag_chain,
    get_session_history,
    input_messages_key="input",
    history_messages_key="chat_history",
    output_messages_key="answer",
)

while True:
    query = input("Ask a question: ")

    for chunk in conversational_rag_chain.stream(
        {"input": query,},
        config={
            "configurable": {
                "session_id": "demo_1"
            }
        }
    ):
        if answer_chunk := chunk.get("answer"):
            print(f"{answer_chunk}", end="", flush=True)

    print()

Error Message and Stack Trace (if applicable)

Ask a question: How many colors are in rainbow?
Error in RootListenersTracer.on_chain_end callback: KeyError('answer')
Error in callback coroutine: KeyError('answer')
A rainbow typically has seven colors, which are: Red, Orange, Yellow, Green, Blue, Indigo, Violet.</s>
Ask a question: 

Description

Hi,

I am trying to get the answer as stream, the problem is whenever the conversational_rag_chain.stream() is initiating, using an input it is giving the following errors: Error in RootListenersTracer.on_chain_end callback: KeyError('answer') Error in callback coroutine: KeyError('answer') and then the output is printing as intended.

My question is, how can I solve it? I have entered output_messages_key="answer" in the conversational_rag_chain already, so am I doing something wrong or is a bug?

Any little discussion or help is welcome. Thanks in advance.

System Info

System Information

OS: Linux OS Version: #39-Ubuntu SMP PREEMPT_DYNAMIC Fri Jul 5 21:49:14 UTC 2024 Python Version: 3.12.3 (main, Apr 10 2024, 05:33:47) [GCC 13.2.0]

Package Information

langchain_core: 0.2.23 langchain: 0.2.11 langchain_community: 0.2.10 langsmith: 0.1.93 langchain_chroma: 0.1.2 langchain_openai: 0.1.17 langchain_text_splitters: 0.2.2

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langgraph langserve

Error is no more there - upgraded to latest versions as below: langchain : 0.2.14 langchain-community : 0.2.12 langchain-core : 0.2.32

[kangcp]

Error in RootListenersTracer.on_chain_end callback: KeyError('message_history') Error in RootListenersTracer.on_chain_end callback: KeyError('message_history') Error in RootListenersTracer.on_chain_end callback: KeyError('message_history') Error in RootListenersTracer.on_chain_end callback: KeyError('message_history')

After upgrading to the latest version, I still have this problem。

langchain 0.2.14 langchain-anthropic 0.1.23 langchain-aws 0.1.16 langchain-community 0.2.12 langchain-core 0.2.32 langchain-openai 0.1.21 langchain-text-splitters 0.2.2

[agvinaykr]

I am still facing error.. Error in AsyncRootListenersTracer.on_chain_end callback: KeyError('output')

langchain-core==0.2.32 langchain==0.2.14 langchain-community==0.2.12 langchain-openai==0.1.21

[harikt]

@kunwarmahen point of import is not valid.

from langchain.chains.history_aware_retriever import create_history_aware_retriever
from langchain.chains.retrieval import create_retrieval_chain

The module is already pointing to it.

What I noticed is when we do

for chunk in conversational_rag_chain.stream(
        {"input": query,},
        config={
            "configurable": {
                "session_id": "demo_1"
            }
        }
    ):
    print(chunk)

the output_messages_key="answer", is not taken into effect. I noticed strange behaviours

1. When using RAG
2. Without using RAG

When using non RAG, you don't need to pass "output_messages_key="content", . This way you can get the content back without error.

If you are using RAG, you can use output_messages_key="answer", and call either invoke or stream.

In the StreamResponse what I did was somthing like

for item in chain.stream(input, config):
    print("Item :: ", item, type(item))
    # Normal chat
    if isinstance(item, AIMessageChunk) and hasattr(item, 'content') and item.content:  # Check if 'content' exists and is not empty
        response = {"content": ''}
        if runtime == 'ollama':
            response = {"content": item.content, "id": item.id}
        else:
            response = {"content": str(item)}
        yield json.dumps(response) + "\n"  # Add a newline to separate JSON objects

    # rag chat
    if isinstance(item, AddableDict) and 'answer' in item:
        response = {"content": ''}
        if runtime == 'ollama':
            response = {"content": item['answer']}
        else:
            response = {"content": str(item)}
        yield json.dumps(response) + "\n"  # Add a newline to separate JSON objects

[daniel-fudge]

Same issue.

langchain 0.3.0 langchain-aws 0.2.0 langchain-community 0.3.0 langchain-core 0.3.0

Error in RootListenersTracer.on_llm_end callback: KeyError('message') Error in RootListenersTracer.on_llm_end callback: TracerException('No indexed run ID c5ac4e93-8732-464f-8400-6b9469b7d986.')

[gulnuravci]

I am just following the example on Build a Chatbot page and experiencing a similar error WARNING:langchain_core.callbacks.manager:Error in RootListenersTracer.on_llm_end callback: KeyError('message'), which also seems to be preventing the message history from persisting. 🤔

[DanielViglione]

I was also getting the same: Error in RootListenersTracer.on_chain_end callback: KeyError('answer') But it was only for non-rag chain. Removing output_messages_key="answer" for non-rag worked.