Implement an approach to filter existing flows so a subflow could be executed

CoxAutomotiveDataSolutions / waimak

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.

Apache License 2.0

75 stars 16 forks source link

Implement an approach to filter existing flows so a subflow could be executed #49

Open alexjbush opened 5 years ago

alexjbush commented 5 years ago

Implement a filter flow action that would take a list of labels to filter with. All actions related to these labels would be executed, and unrelated actions would be filter out.

Things to think about:

Need to handle tags and dependencies
Could be implemented as a generic flow transform function passed to the executor?