search

shroominic / funcchain

⛓️ build cognitive systems, pythonic
https://shroominic.github.io/funcchain/
MIT License
320 stars 21 forks source link

Async support #5

Closed janvandervegt-db closed 11 months ago

janvandervegt-db commented 11 months ago

Hi,

Came across your package on Twitter and it looks very promising. I'll try it for my use case later today but I want to use it for synthetic data generation - requiring large throughput. Looking through the source code, it's not entirely clear to me how easy it is to run async pipelines with this - or what would be required to get there?

To clarify what I'm trying to do, I have a list of a 1000 structured inputs and want to generate the 1000 corresponding outputs. It's okay if a few fail to parse, but it's not okay if they need to run in sequence. I want it to be agnostic to using OpenAI (testing purposes, where the API calls would be async) or local LLMs (where the inputs should be batched in the Hugging Face model interface).

I'm not an expert on LangChain or on asynchronous Python but happy to contribute if funcchain is a good fit for my project.

shroominic commented 11 months ago

For async instead of importing chain you need to import achain. from funcchain import achain Then you can create an async function and you do return await achain(). For parallel compute asyncio.gather is a really good for this kind of tasks. Let me know if the works for you :)