potash / drain

pipeline library
MIT License
12 stars 5 forks source link

Support adaptive workflows #38

Open potash opened 7 years ago

potash commented 7 years ago

Once we've switched to make (#37) we can support adaptive workflows using recursive make. The prototypical use case is feature search, e.g. adding/removing one feature (or set of features) at a time according to some metric.

potash commented 7 years ago

One option for implementation is to add a Workflow class which extends Step but returns a list of steps. When generating the Makefile, the rule corresponding to a workflow gets executed (like any step) and then its result (a list of steps, i.e. the leaves of a new workflow) generate a new Makefile.