threshold tuning with resample

[MaximilianPi]

Hi mlr3 team,

first of all, many thanks for all your work.

I'm currently porting my code from mlr to mlr3. In one of my pipelines I'm tuning hyper parameter with a threshold dependent measurement. In mlr, I can do this by setting the relevant arguments in the TuneControl Object:

tuneCtrl <- mlr::makeTuneControlRandom(maxit = 10L, tune.threshold = TRUE, tune.threshold.args = list(measure = acc))

How do I port this to mlr3 (in a resample strategy)? Here a minimal working example:

library(mlr3)
library(mlr3tuning)
library(paradox)

task = tsk("iris")
pars = ParamSet$new(list(ParamInt$new("mtry", 1, 3)))
learner = lrn("classif.ranger")

terminator = trm("evals", n_evals = 2L)
tuner = tnr("random_search")

tune_learner = AutoTuner$new(learner = learner, 
                             terminator = terminator, 
                             search_space = pars, 
                             tuner = tuner,
                             measure = msr("classif.acc"),
                             resampling = rsmp("cv", folds = 2L))

result = resample(task, tune_learner, rsmp("cv", folds = 2L))

[pfistfl]

I started to write a gallery post on this topic:
https://github.com/mlr-org/mlr3gallery/pull/73/files Hope this already helps!

[MaximilianPi]

Great!

The section about the "PipeOpTunethreshold" is exactly what I need!

Thanks.