search

zygmuntz / hyperband

Tuning hyperparams fast with Hyperband
http://fastml.com/tuning-hyperparams-fast-with-hyperband/
Other
591 stars 75 forks source link

Optimize with CV and specific scoring #5

Closed psinger closed 7 years ago

psinger commented 7 years ago

Hi! I am wondering whether it is possible to optimize with cross validation and preferably with a custom scoring function. Currently, it picks the configuration that minimizes e.g., the log loss of the training data if I am not mistaken. Would be good to also have similar options as grid search offers in scikit learn.

zygmuntz commented 7 years ago

Hi! It is straightforward to use another loss function. For example, when using a scikit-learn compatible classifier, see this: https://github.com/zygmuntz/hyperband/blob/master/common_defs.py#L63

You could also modify that very function to use cross-validation.

psinger commented 7 years ago

Thanks, already figured it out :)