Feature/continue - Githubissues

What?

Implements "Continue Generating" button functionality (akin to ChatGPT's)

Why?

Make it simple to continue generating and reach feature parity with other chat-based experiences?

How?

At a high level

Track output tokens
Compare output tokens to max tokens + return boolean for match
If max tokens, render "Continue Generating" button.

Output token tracking

Stores token data in the LangChain AIMessageChunk model's metadata.

Use a new ChunkWithCount model to track the content and token count of each output chunk. This allows for more granular tracking of the generated output.

Integrates aforementioned model into _raw_output_for_explicit_tool_calling_llms which is used by all LLM streaming methods.

Total token count

While processing stream in extract_citations_from_stream, count up tokens and compare to the current maximum (logic within the DanswerAnswerPiece model.

Display

The React checks stream for optional parameter max_token and displays "Continue generating" component if the value is true

UI

UI was implemented using existing component formatting to maintain style consistency. Example below.

danswer-ai / danswer

Feature/continue #1571