What happened to Pipeline

[mrocklin]

The Pipeline class seems to have disappeared. Are we now just introspecting sklearn pipelines to avoid recomputations of earlier stages?

[jcrist]

Yes, all the wrapper classes have been removed. A few reasons:

• Due to mutation being a core part of the scikit-learn api, they clashed a bit when used together with other scikit-learn things, or the grid search objects (which aren't lazy and do mutate)
• The above also made them hard to maintain.
• Didn't seem that useful compared to the *SearchCV classes
• To fully support all the parameters in the *SearchCV classes, we need to generate graphs more complicated than were done by using the wrapper classes. For example, to support error_score we need to keep track of the error state in the pipeline, something that the wrappers couldn't (and shouldn't imo) do.

All this was done in https://github.com/dask/dask-learn/pull/14, if you want to see the commits.

[mrocklin]

Can you verify that we're still avoiding re-computation of early stages in a pipeline? Quick performance tests seem to suggest this.

[jcrist]

Yes. Both Pipeline and FeatureUnion are broke into components, and duplicate computations aren't repeated.