Pipeline configuration management

kungfuai / kaishi

Tool kit to accelerate exploratory data analysis and data cleaning

https://kaishi.readthedocs.io/en/latest/

MIT License

11 stars 2 forks source link

Closed mwharton3 closed 4 years ago

mwharton3 commented 4 years ago

It's likely that a pipeline would vary often. Having a method like dataset.configure() that guides the user through the process would be useful.

mwharton3 commented 4 years ago

Per @zzsi looking in the sklearn pipeline code could spark some good ideas

mwharton3 commented 4 years ago

Completed. A general guide:

Pipelines are actor objects that hold components
Each component inherits from the kaishi.core.pipeline_component.PipelineComponent class
When a pipeline component is initialized and added to the pipeline, the only argument is a dataset that it stores as a member
A default "configure" method is inherited, but if custom configuration is needed, the child class should create a new one with only named arguments (the config defaults)