After new parameter selection in an evo algo, allow a cleanup/reject operation - Githubissues

ContinuumIO / elm

Phase I & part of Phase II of NASA SBIR - Parallel Machine Learning on Satellite Data

http://ensemble-learning-models.readthedocs.io

43 stars 27 forks source link

After new parameter selection in an evo algo, allow a cleanup/reject operation #205

Open PeterDSteinberg opened 6 years ago

PeterDSteinberg commented 6 years ago

In hyperparameter searches a common problem is an algorithm that selects combos of parameters that may be:

Invalid for the estimator being used
Improbable for the parameters given what is known for the parameters beforehand

One fix for EaSearchCV (and RandomizedSearchCV) would be to have the __init__ arguments allow:

post_select: Function taking the next parameter set as an argument, returning False/None if the next parameter set should be skipped, else a dict that can go to estimator.set_params()

TODO for this issue:

Create the post_select arg for RandomizedSearch, EaSearchCV
Notebook showing how to use post_select in EaSearchCV
Documentation in notebook/elsewhere linking several ways of controlling how combinations of parameters may be avoided / favored, including options already in scikit-learn with ParameterGrid