When I use ConversationalRetrievalChain.from_llm to implement a knowledge base with context, the resulting stream will carry questions, so how can I remove the questions?

whm233 commented 7 months ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.

Example Code

llm = ChatOpenAI( streaming=True, callbacks=[StreamingStdOutCallbackHandler()], temperature=0, openai_api_key="xxx", openai_api_base="http://0.0.0.0:8000/v1/" ,)

qa_chain = ConversationalRetrievalChain.from_llm( llm= llm, retriever= compression_retriever, chain_type='stuff', combine_docs_chain_kwargs = chain_type_kwargs ) a = qa_chain( { "question": question, "chat_history": chat_history, "output_key": 'answer', }, )

Error Message and Stack Trace (if applicable)

.

Description

When I type in a question, it always repeats the question in the answer like this : question :'电子票据查验平台如何获取票据明细？' answer:'电子票据查验平台如何获取票据明细？电子票据查验平台只能查验票据信息，没有票据明细，如需票据明细，请联系开票单位。'

System Info

.0

dosubot[bot] commented 7 months ago

🤖

嗨，@whm233！很高兴再次看到你在这里提问。希望这一次我们也能一起顺利解决问题。

Based on the context provided, it seems like the ConversationalRetrievalChain.from_llm method is repeating the question in the answer because the rephrase_question attribute is set to True by default. This attribute controls whether or not to pass the newly generated question to the combine_docs_chain. If True, it will pass the new generated question along. If False, it will only use the new generated question for retrieval and pass the original question along to the combine_docs_chain.

To prevent the question from being repeated in the answer, you can set the rephrase_question attribute to False when initializing the ConversationalRetrievalChain. Here's how you can do it:

qa_chain = ConversationalRetrievalChain.from_llm(
    llm= llm,
    retriever= compression_retriever,
    chain_type='stuff',
    rephrase_question=False,  # Set this to False
    combine_docs_chain_kwargs = chain_type_kwargs
)

Please try this and let me know if it solves your issue. If you have any other questions or need further clarification, feel free to ask.

For more information, you can refer to the ConversationalRetrievalChain class in the LangChain codebase.

Best regards, Dosu

Sources

libs/langchain/langchain/chains/conversational_retrieval/base.py

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

whm233 commented 7 months ago

rephrase_question``False``ConversationalRetrievalChain

it not use

woojuini commented 4 months ago

I'm suffering from the same problem. rephrase_question=False does not work.

qa_chain = ConversationalRetrievalChain.from_llm(
                    llm=llm,
                    verbose=True,
                     ...
                    combine_docs_chain_kwargs={
                        "prompt": PROMPT
                        },
                    return_source_documents=True,
                    rephrase_question=False
                )

`

whm233 commented 4 months ago

I'm suffering from the same problem. rephrase_question=False does not work.

qa_chain = ConversationalRetrievalChain.from_llm(
                    llm=llm,
                    verbose=True,
                     ...
                    combine_docs_chain_kwargs={
                        "prompt": PROMPT
                        },
                    return_source_documents=True,
                    rephrase_question=False
                )

`

you can do like this llm = ChatOpenAI( streaming=True, callbacks=[StreamingStdOutCallbackHandler()], temperature=0, openai_api_key="xxx", openai_api_base="http://0.0.0.0:8000/v1/", ) llm1 = ChatOpenAI( temperature=0, openai_api_key="xxx", openai_api_base="http://0.0.0.0:8000/v1/", )

question_generator = LLMChain(llm=llm1, prompt=CUSTOM_QUESTION_PROMPT)

doc_chain = load_qa_chain(llm, chain_type="stuff", prompt=QA_PROMPT) qa_chain = ConversationalRetrievalChain( retriever=compression_retriever, combine_docs_chain=doc_chain, question_generator=question_generator) You need to separate the chains for streaming output from those for non-streaming output, and then combine them together

xxonyuk commented 2 months ago

me too, still same problem occurs. anyone reslove this problem?

langchain-ai / langchain