It seems we have converged on a solution for rewriting Pipeline in a simpler way, removing ParamTable and using map-reduce operations instead. The current status of the rewrite/prototype can be found in the v2-prototype branch. We will likely want to distribute the work on this soon, so here is a summary of the status, which I intend to keep up to date.
Last update: 2024-04-15
To do
Cyclebane needs an overhaul, more testing, code reviews, and a release.
Several features that would be required for parity with Sciline's current features via ParamTable are not implemented (in particular: groupby), but I think that is ok for now. We are not actually using this anywhere (aside from docs and tests in Sciline).
Add dependency on networkx. Also have a brief discussion on whether we want to do this.
Pipeline.visualize vs. DataGraph.visualize, do we want both?
Pipeline._repr_html_
Fix to-dos in code, test DataGraph in case there are features the old Pipeline tests do no cover.
Rewrite/remove tests of Pipeline using ParamTable
Rewrite docs
Open questions
Do we have consensus that making constraints on type vars mandatory is ok, or do we need to look for a better solution?
Should we have DataGraph, underlying Pipeline, or should it be the same.
It seems we have converged on a solution for rewriting
Pipelinein a simpler way, removingParamTableand usingmap-reduceoperations instead. The current status of the rewrite/prototype can be found in thev2-prototypebranch. We will likely want to distribute the work on this soon, so here is a summary of the status, which I intend to keep up to date.Last update: 2024-04-15
To do
ParamTableare not implemented (in particular:groupby), but I think that is ok for now. We are not actually using this anywhere (aside from docs and tests in Sciline).networkx. Also have a brief discussion on whether we want to do this.Pipeline.visualizevs.DataGraph.visualize, do we want both?Pipeline._repr_html_DataGraphin case there are features the oldPipelinetests do no cover.PipelineusingParamTableOpen questions
DataGraph, underlyingPipeline, or should it be the same.