0ptim / JellyChat

🪼 AI chatbot for the DeFiChain ecosystem.
https://defichainwiki.com/jellychat
MIT License
11 stars 2 forks source link

Stream Final Agent Response #45

Closed 0ptim closed 1 year ago

0ptim commented 1 year ago

Description

We want to stream the tokens from the final output of the agent.

This allows the user to start reading the response as soon as it starts to get generated, instead of waiting until the whole answer is done.

This should greatly improve the time until a response is visible to the user, especially for longer answers. It would also make the bot feel more “alive”, since it would feel more like someone who's typing in real-time.

Implementation

Todo's

Resources