Add support for irace v4

[MLopez-Ibanez]

Hi,

The next version of irace: https://github.com/MLopez-Ibanez/irace/tree/v4 will have some breaking changes: https://mlopez-ibanez.github.io/irace/news/index.html

The list is not complete yet but among them, there is the call to irace() here:

https://github.com/mlr-org/bbotk/blob/70f3f96ff0023066aa8eaa7c97484f0d6e518663/R/OptimizerIrace.R#L182-L185

that should be replaced by:

      scenario = c(list(maxExperiments = terminator$param_set$values$n_evals, targetRunnerData = list(inst = inst), parameters = paradox_to_irace(inst$search_space, pv$digits)), pv)
      # run irace
      res = invoke(irace::irace, scenario = scenario, .opts = allow_partial_matching)

Also, a few lines below, instead of load() you can use iraceResults = read_logfile() introduced in irace 3.5

[be-marc]

Hey, thanks for the information. I assume you are uploading to CRAN and we have 2 weeks to upload bbotk with the changes? If there are any more changes please let me know.

[MLopez-Ibanez]

It will still take some time to reach CRAN. Probably not before June. Other changes that may be relevant to you:

• Requires R version >= 3.6.0.

• Logfiles *.Rdata use format version 3, which can only be read by R version >= 3.5.

• The scenario options forbiddenFile and forbiddenExps have been removed and will give an error if present. Forbidden configurations are now specified in the parameter space description. See the example in readParameters().

• The scenario option digits has been removed and will give an error if present. The number of digits for real-valued parameters is now specified in the parameter space description. See the example in readParameters().

• The scenario object now includes the parameters object. Thus functions such as irace(), which previously took as arguments both scenario and parameters, now only take scenario. This also means that the log file irace.Rdata does not contain a separate parameters element since this element can now be found within scenario.

• irace warns about using '&&' and '||' instead of '&' and '|' in parameter conditions and forbidden expressions. A future version of irace will reject those uses as errors.

• The column "instance" of the instancesList data frame stored in the logFile has been renamed to "instanceID". This data frame should not be accessed directly. Instead use the new function get_instanceID_seed_pairs().

• irace is now more strict in enforcing runtime bounds given with scenario$boundMax and will stop with an error if the target-runner reports a runtime larger than the given bound.

• All functions that contained a period ('.') in the name have been renamed to use '_' instead.

[MLopez-Ibanez]

I am doing the final tests before uploading irace v4 to CRAN. This may still take me at least a few days, more if I find bugs. A non-exhaustive list of changes:

• [x] parameters is now stored in scenario: https://github.com/mlr-org/bbotk/blob/70f3f96ff0023066aa8eaa7c97484f0d6e518663/R/OptimizerIrace.R#L182-L185

    scenario = c(list(parameters = paradox_to_irace(inst$search_space, pv$digits), maxEperiments = terminator$param_set$values$n_evals, targetRunnerData = list(inst = inst)), pv)
    res = invoke(irace::irace, scenario = scenario, .opts = allow_partial_matching)

• [x] Replace $experimentLog with $state$experiment_log. And it is already a data.table: https://github.com/mlr-org/bbotk/blob/70f3f96ff0023066aa8eaa7c97484f0d6e518663/R/OptimizerIrace.R#L190

• [x] Use iraceResults = irace::read_log(self$param_set$values$logFile) to replace: https://github.com/mlr-org/bbotk/blob/70f3f96ff0023066aa8eaa7c97484f0d6e518663/R/OptimizerIrace.R#L188-L189

• [x] We have replaced '.' with '_' in user-visible functions and argument names. So the arguments exec.target.runner and target.runner should be adjusted.

• [x] The scenario option digits has been removed and will give an error if present. The number of digits for real-valued parameters is now specified in the parameter space description. See the example in readParameters(). Alternatively, you may create the parameters programmatically using the new API: https://github.com/MLopez-Ibanez/irace/blob/097696f8a9b18de4cf04a7b10897a3a77c7a1814/R/parameters.R#L238

• [ ] Note that irace supports logscale parameters and default values (given as initial configurations), as explained in the above links.

There may be other things that I have missed. If you find any problems or crashes, please let me know. It would be great to fix them before uploading to CRAN.

[MLopez-Ibanez]

You can install the next version of irace from git: https://github.com/MLopez-Ibanez/irace

This version will go to CRAN in the next few weeks and break bbtok.

[be-marc]

Thanks for all the information. #243 contains the changes. We will upload when CRAN informs us. The installation of your packages somehow does not work with pak::pak("MLopez-Ibanez/irace"). So our ci fails but devtools::test() works locally.

[MLopez-Ibanez]

I fixed the problem and pak install should work now.

[be-marc]

Done