mlr-org / mlr3hyperband

Successive Halving and Hyperband in the mlr3 ecosystem
https://mlr3hyperband.mlr-org.com
GNU Lesser General Public License v3.0
18 stars 5 forks source link

Bug: Hyperband does not work correctly with CV-resampling #50

Closed juliambr closed 4 years ago

juliambr commented 4 years ago

I ran hyperband once with holdout as resampling, and once with 5-fold cross-validation. For the cross-validation, the wrong individuals "survive", which is exemplarily visualized for a single bracket:

error_plot_pima

Reason for this is this line of code, which should be instance$bmr$aggregate.