search

mlr-org / mlr3hyperband

Successive Halving and Hyperband in the mlr3 ecosystem
https://mlr3hyperband.mlr-org.com
GNU Lesser General Public License v3.0
18 stars 5 forks source link

tuner$info slot should be stored in bmr$rr_data #29

Closed berndbischl closed 4 years ago

berndbischl commented 5 years ago

mlr3 now has a dedicated data structure for this, pls use this ASAP

berndbischl commented 5 years ago

https://github.com/mlr-org/mlr3/blob/master/R/BenchmarkResult.R

please read the section on rr_data

this will be auto-joinded with inst$archive

SebGGruber commented 5 years ago

so, should tuner$info also be removed then? I'm kind of against this. Because rr_data is for every resampling evaluation, while tuner$info only for every bracket stage, which means we will have a lot of entries from the latter being repeated very often in the former, resulting in a more cluttered bracket layout overview. Additionally, if I would delete the info attribute, where should I put the docs about the columns then?

SebGGruber commented 5 years ago

only expanded inst$rr_data so far (tuner$info still there):

  set.seed(123)

  ps = ParamSet$new(list(
    ParamInt$new("nrounds", lower = 1, upper = 8, tags = "budget"),
    ParamFct$new("booster", levels = c("gbtree", "gblinear", "dart"))
  ))

  inst = TuningInstance$new(
    tsk("iris"),
    lrn("classif.xgboost"),
    rsmp("holdout"),
    msr("classif.ce"),
    ps,
    term("evals", n_evals = 100000)
  )

  tuner = TunerHyperband$new(eta = 2L)
  tuner$tune(inst)

  inst$bmr$rr_data
jakob-r commented 4 years ago

All info now resides in the OptimInstance$archive