How to set `grid_search` in the order as appeared in `generate_design_grid()`?

[Fred-Wu]

Hi, I have two questions related to tuning process.

1. How can I set grid_search tuning in the order as appeared in generate_design_grid()?

2. Since the grid_search seems to run at random, if the terminator is set as stagnation, would the tuning just stop based on the random search order?

[be-marc]

How can I set grid_search tuning in the order as appeared in generate_design_grid()?

Just create a fixed design. You can sort the design data.table in any order.

library("mlr3tuning")

search_space = ps(
  minsplit = p_int(2, 128, logscale = TRUE), 
  minbucket = p_int(1, 64, logscale = TRUE),
  cp = p_dbl(1e-04, 1e-1, logscale = TRUE)
)

design = generate_design_grid(search_space, resolution = 5)$data

instance = tune(
  method = "design_points",
  task = tsk("pima"),
  learner = lrn("classif.rpart"),
  resampling = rsmp("cv", folds = 3),
  measure = msr("classif.ce"),
  search_space = search_space,
  batch_size = 5,
  design = design
)

Since the grid_search seems to run at random, if the terminator is set as stagnation, would the tuning just stop based on the random search order?

Yes.

[mllg]

I'll introduce an option to allow disabling the randomization of experiments in mlr3.

[be-marc]

Yes, mlr3 also randomizes the order within a batch but grid_search itself randomizes the grid.

[mllg]

Can we make the shuffling in TunerGridSearch a hyperparameter?