arcalot / arcaflow-engine

Arcaflow is a highly-portable workflow engine enabling modular and validated pipelines through containerized plugins.
https://arcalot.io/arcaflow/
Apache License 2.0
6 stars 9 forks source link

Implement wait_for functionality in foreach loop workflow steps #125

Closed dustinblack closed 5 months ago

dustinblack commented 7 months ago

Please describe what you would like to see in this project

The wait_for functionality allows the running of a step to block on output from another step. Currently this functionality only work for discrete plugin steps. I would like to be able to use this same functionality on foreach loop workflow steps, blocking a loop step on the output of another loop or plugin step.

Please describe your use case

Complex workflows will require serialization of sub-workflows so that actions are performed in a deterministic order and without one set of operations potentially interfering with another. For instance, I may want to run a loop of benchmarks of one type, and the only when that loop completes I would run a loop of benchmarks of another type.

dustinblack commented 5 months ago

This works in the engine today. Likely resolved with #128