mlr-org / mlr3tuning

Hyperparameter optimization package of the mlr3 ecosystem
https://mlr3tuning.mlr-org.com/
GNU Lesser General Public License v3.0
53 stars 5 forks source link

Create search_space from TuneTokens #285

Closed be-marc closed 3 years ago

jakob-r commented 3 years ago

@be-marc Can you check if adaptation are needed here: https://github.com/mlr-org/mlr3book/pull/209 (and in case make a PR to this branch or talk to @mb706 and edit directly)

be-marc commented 3 years ago
learners = list(
  lrn("classif.kknn", id = "kknn"),
  lrn("classif.svm", id = "svm", type = "C-classification")
)

graph =
  po("branch", options = c("kknn", "svm")) %>>%
  gunion(lapply(learners, po)) %>>%
  po("unbranch")

graph_learner = GraphLearner$new(graph)

graph_learner$param_set$values$branch.selection = to_tune(c("kknn", "svm"))

instance = TuningInstanceSingleCrit$new(
  task = tsk("iris"),
  learner = graph_learner,
  resampling = rsmp("cv", folds = 3),
  measure = msr("classif.ce"),
  terminator = trm("evals", n_evals = 60)
)

Fails since we try to remove the TuneToken from the parameter set but branch.selection is a required parameter (TuningInstanceSingleCrit.R - 128).

be-marc commented 3 years ago

I am dissatisfied with the solution to store a learner with TuneToken and the associated search space in ObjectiveTuning. We store the same information and have an unsable learner again. We should discuss a clean solution in the future #286. For now, TuneTokens are removed in ObjectiveTuning$.eval_many to avoid errors with required parameters.