run-llama / llama_index

LlamaIndex is a data framework for your LLM applications
https://docs.llamaindex.ai
MIT License
35.37k stars 4.98k forks source link

[Feature Request]: Create a stream agent + workflows example #15393

Open logan-markewich opened 4 weeks ago

logan-markewich commented 4 weeks ago

Feature Description

How can users stream while accounting for tool calls in a workflow? Currently, the way I would implement this is not very friendly or example-ready

Reason

Streaming is a super popular feature, we should have a way to do this while detecting tool calls

Value of Feature

No response

nerdai commented 2 weeks ago

@logan-markewich do you mind providing details on how you might implement this today, plus maybe some high-level suggestions on how it should ideally be done?

Thinking this might be a good and useful thing for me to tackle.

logan-markewich commented 2 weeks ago

@nerdai we have two existing agent examples in the docs

To add streaming would be basically copying what the existing openai agent or react does for streaming for example

However, this doesn't make for great example, because it's super complicated 😅 i was hoping to be able to implement some util function to make this less scary, or possibly even refactor how we do streaming to make it more straightforward