dont call without_logging pls

[berndbischl]

it really not good for debugging and so on to do this internally the logger can be auto-configured from the outside to achieve the same

[berndbischl]

https://github.com/mlr-org/mlr3book/issues/56

[mb706]

The problem here is that the resampling evaluations print a lot of info logs that drown out the (few) info logs of mlr3hyperband. There is probably a way to do fine-grained settings here, like

# switch out most logging
lgr::get_logger("mlr3")$set_threshold("warn")
# switch on mlr3tuning logs specifically
lgr::get_logger("mlr3/mlr3tuning")$set_threshold("info")

Maybe we want to have a setting where, by default, resample() prints lots of resampling logs, but tuner$tune() suppresses the individual resampling logs (which are less interesting here because not top level any more). I guess there is a way for the user to do that specifically (see code snippet above) but it would be a useful result.

@mllg do you have any ideas here?

[SebGGruber]

Current solution: During package loading, a new level ("info hb" = 350) between "info" and "warn" is added to the "mlr3/mlr3tuning" logger. Furthermore, the threshold is then set to "info hb", ignoring all "info" logs of ml3tuning. Also, the threshold of the logger "mlr3" is set to "warn". An alternative solution would be to create an additional logger "mlr3/mlr3tuning/mlr3hyperband" and set the other two loggers to "warn". But I have the feeling it would be a bit messy to work with 3 separate loggers.

Example: (branch seb-dev):

set.seed(123)

ps = ParamSet$new(list(
  ParamInt$new("nrounds", lower = 1, upper = 8, tags = "budget"),
  ParamFct$new("booster", levels = c("gbtree", "gblinear", "dart"))
))

inst = TuningInstance$new(
  tsk("iris"),
  lrn("classif.xgboost"),
  rsmp("cv", folds = 2),
  msr("classif.ce"),
  ps,
  term("evals", n_evals = 100000)
)

tuner = TunerHyperband$new(eta = 2L)
tuner$tune(inst)

results = inst$archive()[, .(
  nrounds = sapply(params, "[", "nrounds"),
  booster = sapply(params, "[", "booster"),
  classif.ce
)]

[mb706]

Interesting solution, I think @berndbischl or @mllg should comment if this is okay.

[berndbischl]

we decided to have a dedicated mlr3hyperband logger, users can then simply switch off the mlr3 or mlr3tuning logger

[jakob-r]

We decided to only have one bbotk logger for mlr3tuning etc. We can revise this later on quite easily. But i will close for now.