Hyperparameter selection and Feature Selection Simultaneously

[samyip123]

Does the Evaluator perform model selection and feature selection simultaneously, choosing the hyperparameter of lower layers simultaneously with feature and preprocessing selecting for each classifier, letting them have different features and different preprocessing pipelines. The example in the tutorial seems to only allow choosing parameters in the lower layers , i may be mistaken, thanks

[flennerhag]

Hi,

I'm not sure I fully understand your question, but anything in the preprocessing stack is not tuned for hyper-parameters. For instance, if you use lower-level layers of an ensemble for preprocessing, these parameters are frozen.

On the other hand, if you include the lower levels as part of the model to perform model selection over, you would be tuning both the lower levels and the higher levels.

Note that this can be very costly since you have an outer CV loop in the model selection operator, as well as a CV loop in each model you're fitting (since you re-fit the lower levels for outer fold).

Hope this answers your question; if not, let me know!