Pipeline run logic not robust with cyclic graphs

deepset-ai / haystack

:mag: AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

https://haystack.deepset.ai

Apache License 2.0

16.7k stars 1.83k forks source link

Pipeline run logic not robust with cyclic graphs #8024

Open shadeMe opened 2 months ago

shadeMe commented 2 months ago

Follow-up to https://github.com/deepset-ai/haystack/issues/7985, related to https://github.com/deepset-ai/haystack/issues/7960

The workaround that was added to "fix" the above issue only applies to pipeline graphs that are DAGs. As it currently stands, pipelines containing loops/cycles can cause components to misfire/fire more than once and/or launch in an indeterminate order.

Possible contributing factors

Support for default inputs.
Variadic (greedy) components.

shadeMe commented 2 weeks ago

Experimental fix in https://github.com/deepset-ai/haystack/tree/subgraphs