search

Chainlit / cookbook

Chainlit's cookbook repo
https://github.com/Chainlit/chainlit
726 stars 270 forks source link

Answers of follow up questions are not streamed (sample of pdf-qa) #25

Closed kay-schmitteckert closed 12 months ago

kay-schmitteckert commented 1 year ago

Hi all,

first of all thanks for all the cookbooks - really great work and helpful! In the sample app pdf-qa (https://github.com/Chainlit/cookbook/tree/main/pdf-qa), the first question is answered in a proper way but if it comes to follow up questions, it streams just the "rephrasing" of the question instead of the final answer.

Is there any possibility to overcome this?

Best Kay

willydouhard commented 1 year ago

Hello, it seems that it's not possible to use final answer streaming with the conversational chain because:

  1. It does not use an answer prefix
  2. It can take more than one step

So either use a retrieval chain (which always take 1 step i believe) or just disable the final answer streaming.

I will update the examples after Wednesday's release.