[Bug]: Response duplicated

hunterbmt commented 3 months ago

What happened?

A bug happened!

after pull the lastest version, brain start to give out duplicate result

Relevant log output

No response

Twitter / LinkedIn details

No response

linear[bot] commented 3 months ago

TECH-68 [Bug]: Response duplicated

dosubot[bot] commented 3 months ago

Hey there, @hunterbmt! I'm here to assist you with any bugs, questions, or contributions you might have. Let's tackle this issue together!

The issue of response duplication could be related to several recent changes in the codebase:

Caching Mechanism: A caching mechanism was introduced to optimize performance by reducing redundant calls, which might be causing the same response to be served multiple times ^[1].
Query Processing and Matching: Recent changes in the query processing and matching system, including fixes to integrations and refactoring of the conversational QA chain initialization, could potentially lead to duplication ^[1]^[2].
BigBrain Class Updates: The BigBrain class has seen updates, including the addition of the generate_stream method for real-time response generation. This method constructs a conversational QA chain and processes chat history and questions, which might contribute to duplication if not handled correctly ^[2].
Prompt Templates and Response Formatting: Recent changes in the prompt templates or response formatting could inadvertently cause responses to be duplicated. The generate_answer and generate_stream methods use the ChatPromptTemplate.from_messages method to format the prompt, and improper handling of the prompt_formated variable could lead to duplicated responses ^[3].

Reviewing these areas in the codebase might help identify and resolve the duplication issue.

To continue talking to Dosu, mention @dosu.

hunterbmt commented 3 months ago

@dosu not helpful at all

dosubot[bot] commented 3 months ago

@StanGirard, this one's a bit out of my wheelhouse, could you step in and provide your guidance?

To continue talking to Dosu, mention @dosu.

hunterbmt commented 3 months ago

@AmineDiro any idea on this issue ? I confirm it's not FE issue

StanGirard commented 3 months ago

@hunterbmt We haven't changed anything. openai might have push a new update with a bug. I'll look deeper into it :/

hunterbmt commented 3 months ago

@StanGirard if it help, I make some debugging myself, for some reason, once a while OpenAI will send back a chunk that make tool_calls in gathered_msg empty, therefore the prev_answer in ultis.py/parse_chunk_response has 0 len => duplicate in our logic (since we depend on prev_answer to indicate what is the new answer chunk)

StanGirard commented 3 months ago

@AmineDiro might be a bug

StanGirard commented 3 months ago

@hunterbmt would love if you could open a PR

zdkderpphb commented 2 months ago

@hunterbmt This problem has also been bothering me for the last few days. In particular, it lies in get_prev_message_str. If gathered_msg has no answer attribute, prev_answer is overwritten with an empty string. This results in len(prev_answer)==0 and this leads to the entire content being listed in answer_str again instead of just the newly added content. I have temporarily solved it with a global variable prev_answer and this should then only be overwritten if get_prev_message_str is not empty. Quick and dirty :)

prev_answer = ""

def parse_chunk_response(
    gathered_msg: AIMessageChunk,
    raw_chunk: dict[str, Any],
    supports_func_calling: bool,
) -> Tuple[AIMessageChunk, ParsedRAGChunkResponse]:
    global prev_answer
    # Init with sources
    answer_str = ""
    # Get the previously parsed answer
    new_prev_answer = get_prev_message_str(gathered_msg)
    # Check if the new_prev_answer is empty, if so, use the old prev_answer
    if new_prev_answer:
        prev_answer = new_prev_answer

AmineDiro commented 2 months ago

Hello @zdkderpphb thanks for taking the time for digging into the bug !

I fixed this in a971e90 :). Everything should run smoothly now !

zdkderpphb commented 2 months ago

[like] Michael Leitgeb reacted to your message:

From: AmineDiro @.> Sent: Thursday, July 11, 2024 1:21:29 PM To: QuivrHQ/quivr @.> Cc: Michael Leitgeb @.>; Mention @.> Subject: Re: [QuivrHQ/quivr] [Bug]: Response duplicated (Issue #2794)

Hello @zdkderpphbhttps://github.com/zdkderpphb thanks for taking the time for digging into the bug !

I fixed this in a971e90https://github.com/QuivrHQ/quivr/commit/a971e90f4d7f135852e4ec000b6a7769933e5ebd :). Everything should run smoothly now !

— Reply to this email directly, view it on GitHubhttps://github.com/QuivrHQ/quivr/issues/2794#issuecomment-2222927699, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A6DDXCKUFQ252ENUHNWKYI3ZL2BFTAVCNFSM6AAAAABKJON3E2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRSHEZDONRZHE. You are receiving this because you were mentioned.Message ID: @.***>

QuivrHQ / quivr