Streaming partial messages for UI integration.

microsoft / autogen

A programming framework for agentic AI 🤖

https://microsoft.github.io/autogen/

Creative Commons Attribution 4.0 International

33.73k stars 4.88k forks source link

Streaming partial messages for UI integration. #3862

Open jackgerrits opened 3 weeks ago

jackgerrits commented 3 weeks ago

We need to ensure the streaming use case is adequately supported in the API. While agent-agent communication likely does not need streaming, we need to ensure agent-UI supports streaming.

User can:

Create a UI which can show partial model output
Interrupt an in progress streaming response

ekzhu commented 3 weeks ago

We should use a closure agent to collect the partial messages and pump them through to the front end via a websocket.

The partial message type is application defined.

We can show case this using a cookbook or a sample directory.

ekzhu commented 2 weeks ago

Related to #3970 and #3969, a simple solution is to just have the agent and team return a streaming iterator.