The current setup of FLoRIN is powerful, but it suffers from enforcing a roughly linear path through the operations, i.e. each op can only access the output of the previous op and the args/kwargs from setup. The next step with FLoRIN is to implement operations/data flow as a directed graph that allows routing data to arbitrary subsequent operations.
To Do
Reformulate FLoRIN using a graph construction/traversal package (networkx?)
Enable serializing/deserializing pipelines preserving only the data necessary to run the pipeline
Introspection to identify data dependencies at runtime
Possibly new semantics for defining and statically parameterizing ops
jeffkinnison
commented
4 years ago
The current setup of FLoRIN is powerful, but it suffers from enforcing a roughly linear path through the operations, i.e. each op can only access the output of the previous op and the args/kwargs from setup. The next step with FLoRIN is to implement operations/data flow as a directed graph that allows routing data to arbitrary subsequent operations.
To Do
networkx?)