search

JoshuaC215 / agent-service-toolkit

Full toolkit for running an AI agent service built with LangGraph, FastAPI and Streamlit
https://agent-service-toolkit.streamlit.app
MIT License
247 stars 40 forks source link

Switch to astream_events to support token-level streaming with Gemini #32

Open peterkeppert opened 1 week ago

peterkeppert commented 1 week ago

Draft PR for #30

JoshuaC215 commented 1 week ago

I think the CI issue is due to lint running on python 3.9 and anext() being added in 3.10. I can fix that, it seems reasonable to set that minimum version.

Will take a closer look at this in the next few days.

JoshuaC215 commented 1 day ago

Made some progress on cleaning this up but there is still some weirdness with filtering the on_chain_end events - seems like the langchain implementation changes across versions, and makes it harder to not leak service details into the agent. I need to take a closer look and think about it some more.

I think with this change we can also remove the asyncio.Queue usage which will simplify things a bit too, which is nice.

I still want to incorporate this but might be later this week or next week before I get a chance to dig in. Also need to do the langchain v0.3 release update (which should solve some other problems too :tada: )