Pipeline Refactoring: Automatic Step Ordering and Dynamic Step Access
Overview:
This refactor introduces a complete overhaul of how pipeline steps are handled within the code. The steps are now automatically ordered to ensure they are always processed in the correct sequence.
New Pipeline Order:
The pipeline steps are now processed in the following order:
Preprocessing
Over/Under Sampling
Feature Selection
Classifier
Key Changes:
Automatic Step Ordering: Each part of the pipeline is type-checked and then automatically sorted into the correct order.
Imputation Before Scaling: We ensure that imputation is performed before scaling during the preprocessing step.
Dynamic Step Naming: Pipeline steps are named according to their function and position in the pipeline. For example, if an imputation step is included, it will be named preprocessing_{step_name}. This convention allows for easier dynamic access to the pipeline components later in the process.
Impact:
Hyperparameter Search: The new naming convention ensures that pipeline components can be accessed dynamically during hyperparameter tuning.
Early Stopping: It is now possible to dynamically access different parts of the pipeline for early stopping.
Feature Selection View: Selected features from the feature selection model can now be accessed dynamically based on the new naming scheme.
Why This Matters:
This refactor improves the robustness and flexibility of the pipeline, making it easier to maintain and extend, especially in workflows involving hyperparameter optimization and model monitoring.
Pipeline Refactoring: Automatic Step Ordering and Dynamic Step Access
Overview:
This refactor introduces a complete overhaul of how pipeline steps are handled within the code. The steps are now automatically ordered to ensure they are always processed in the correct sequence.
New Pipeline Order:
The pipeline steps are now processed in the following order:
Key Changes:
Automatic Step Ordering: Each part of the pipeline is type-checked and then automatically sorted into the correct order.
Imputation Before Scaling: We ensure that imputation is performed before scaling during the preprocessing step.
Dynamic Step Naming: Pipeline steps are named according to their function and position in the pipeline. For example, if an imputation step is included, it will be named
preprocessing_{step_name}. This convention allows for easier dynamic access to the pipeline components later in the process.Impact:
Hyperparameter Search: The new naming convention ensures that pipeline components can be accessed dynamically during hyperparameter tuning.
Early Stopping: It is now possible to dynamically access different parts of the pipeline for early stopping.
Feature Selection View: Selected features from the feature selection model can now be accessed dynamically based on the new naming scheme.
Why This Matters:
This refactor improves the robustness and flexibility of the pipeline, making it easier to maintain and extend, especially in workflows involving hyperparameter optimization and model monitoring.