search

mlr-org / mlr3mbo

Flexible Bayesian Optimization in R
https://mlr3mbo.mlr-org.com
24 stars 1 forks source link

Error in vignettes: `Error: missing value where TRUE/FALSE needed` #160

Closed barracuda156 closed 4 weeks ago

barracuda156 commented 1 month ago 
--->  Testing R-mlr3mbo
Executing:  cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-mlr3mbo/R-mlr3mbo/work/mlr3mbo" && /opt/local/bin/R CMD check ./mlr3mbo_0.2.4.tar.gz --no-manual --no-build-vignettes 
* using log directory ‘/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-mlr3mbo/R-mlr3mbo/work/mlr3mbo/mlr3mbo.Rcheck’
* using R version 4.4.1 (2024-06-14)
* using platform: powerpc-apple-darwin10.0.0d2 (32-bit)
* R was compiled by
    gcc-mp-13 (MacPorts gcc13 13.3.0_0+stdlib_flag) 13.3.0
    GNU Fortran (MacPorts gcc13 13.3.0_0+stdlib_flag) 13.3.0
* running under: OS X Snow Leopard 10.6
* using session charset: UTF-8
* using options ‘--no-manual --no-build-vignettes’
* checking for file ‘mlr3mbo/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘mlr3mbo’ version ‘0.2.4’
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘mlr3mbo’ can be installed ... OK
* used C compiler: ‘gcc-mp-13 (MacPorts gcc13 13.3.0_0+stdlib_flag) 13.3.0’
* used SDK: ‘NA’‘NA’‘NA’‘NA’‘NA’‘NA’
* checking installed package size ... OK
* checking package directory ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking code files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking whether startup messages can be suppressed ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking line endings in C/C++/Fortran sources/headers ... OK
* checking compiled code ... OK
* checking installed files from ‘inst/doc’ ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
  Running ‘testthat.R’
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... OK
* checking running R code from vignettes ...
  ‘mlr3mbo.Rmd’ using ‘UTF-8’... failed
 ERROR
Errors in running code in vignettes:
when running code in ‘mlr3mbo.Rmd’
  ...
INFO  [21:05:20.609] [bbotk]  1 1

> optimizer$surrogate$param_set$values$catch_errors = FALSE

> optimizer$optimize(instance)
INFO  [21:05:20.635] [bbotk] Starting to optimize 1 parameter(s) with '<OptimizerMbo>' and '<TerminatorEvals> [n_evals=10, k=0]'

  When sourcing ‘mlr3mbo.R’:
Error: missing value where TRUE/FALSE needed
Execution halted

* checking re-building of vignette outputs ... SKIPPED
* DONE

Status: 1 ERROR
See
  ‘/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-mlr3mbo/R-mlr3mbo/work/mlr3mbo/mlr3mbo.Rcheck/00check.log’
for details.

Command failed:  cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_R_R-mlr3mbo/R-mlr3mbo/work/mlr3mbo" && /opt/local/bin/R CMD check ./mlr3mbo_0.2.4.tar.gz --no-manual --no-build-vignettes 
Exit code: 1

> library(knitr)

> opts_chunk$set(collapse = TRUE, comment = "#>")

> update_db = function() {
+     if (is.null(db$base) || is.null(db$aliases)) {
+         hdb = hsearch_db(package = unique(c(db$index, db$hosted)), 
 .... [TRUNCATED] 

> ref = function(topic, text = topic, format = "markdown") {
+     strip_parenthesis = function(x) sub("\\(\\)$", "", x)
+     checkmate::assert_strin .... [TRUNCATED] 

> ref_pkg = function(pkg, format = "markdown") {
+     checkmate::assert_string(pkg, pattern = "^[[:alnum:]._-]+$")
+     checkmate::assert_choice(for .... [TRUNCATED] 

> cran_pkg = function(pkg, format = "markdown") {
+     checkmate::assert_string(pkg, pattern = "^[[:alnum:]._-]+$")
+     checkmate::assert_choice(fo .... [TRUNCATED] 

> mlr_pkg = function(pkg, format = "markdown") {
+     checkmate::assert_string(pkg, pattern = "^[[:alnum:]._-]+$")
+     checkmate::assert_choice(for .... [TRUNCATED] 

> gh_pkg = function(pkg, format = "markdown") {
+     checkmate::assert_string(pkg, pattern = "^[[:alnum:]_-]+/[[:alnum:]._-]+$")
+     checkmate::ass .... [TRUNCATED] 

> db = new.env()

> db$index = c("base", "utils", "datasets", "data.table", 
+     "stats")

> db$hosted = c("paradox", "mlr3misc", "mlr3", "mlr3data", 
+     "mlr3db", "mlr3proba", "mlr3pipelines", "mlr3learners", "mlr3filters", 
+     "bbotk ..." ... [TRUNCATED] 

> lgr = NULL

> library(mlr3mbo)

> library(data.table)

> as.data.table(mlr_loop_functions)
Key: <key>
               key                         label    instance
            <char>                        <char>      <char>
1:    bayesopt_ego Efficient Global Optimization single-crit
2:    bayesopt_emo           Multi-Objective EGO  multi-crit
3:   bayesopt_mpcl      Multipoint Constant Liar single-crit
4: bayesopt_parego                        ParEGO  multi-crit
5: bayesopt_smsego                       SMS-EGO  multi-crit
                                  man
                               <char>
1:    mlr3mbo::mlr_loop_functions_ego
2:    mlr3mbo::mlr_loop_functions_emo
3:   mlr3mbo::mlr_loop_functions_mpcl
4: mlr3mbo::mlr_loop_functions_parego
5: mlr3mbo::mlr_loop_functions_smsego

> library(mlr3learners)
Loading required package: mlr3

> surrogate = SurrogateLearner$new(lrn("regr.km"))

> surrogate = srlrn(lrn("regr.km"))

> surrogate$learner
<LearnerRegrKM:regr.km>: Kriging
* Model: -
* Parameters: list()
* Packages: mlr3, mlr3learners, DiceKriging
* Predict Types:  response, [se]
* Feature Types: logical, integer, numeric
* Properties: -

> surrogate$param_set
<ParamSet(4)>
Key: <id>
                     id    class lower upper nlevels        default
                 <char>   <char> <num> <num>   <num>         <list>
1: assert_insample_perf ParamLgl    NA    NA       2 <NoDefault[0]>
2:         catch_errors ParamLgl    NA    NA       2 <NoDefault[0]>
3:         perf_measure ParamUty    NA    NA     Inf <NoDefault[0]>
4:       perf_threshold ParamDbl  -Inf   Inf     Inf <NoDefault[0]>
                parents  value
                 <list> <list>
1:               [NULL]  FALSE
2:               [NULL]   TRUE
3: assert_insample_perf [NULL]
4: assert_insample_perf [NULL]

> as.data.table(mlr_acqfunctions)
Key: <key>
       key                                              label
    <char>                                             <char>
 1:    aei                     Augmented Expected Improvement
 2:     cb                     Lower / Upper Confidence Bound
 3:   ehvi                   Expected Hypervolume Improvement
 4: ehvigh Expected Hypervolume Improvement via GH Quadrature
 5:     ei                               Expected Improvement
 6:   eips                    Expected Improvement Per Second
 7:   mean                                     Posterior Mean
 8:     pi                         Probability Of Improvement
 9:     sd                       Posterior Standard Deviation
10: smsego                                            SMS-EGO
                                 man
                              <char>
 1:    mlr3mbo::mlr_acqfunctions_aei
 2:     mlr3mbo::mlr_acqfunctions_cb
 3:   mlr3mbo::mlr_acqfunctions_ehvi
 4: mlr3mbo::mlr_acqfunctions_ehvigh
 5:     mlr3mbo::mlr_acqfunctions_ei
 6:   mlr3mbo::mlr_acqfunctions_eips
 7:   mlr3mbo::mlr_acqfunctions_mean
 8:     mlr3mbo::mlr_acqfunctions_pi
 9:     mlr3mbo::mlr_acqfunctions_sd
10: mlr3mbo::mlr_acqfunctions_smsego

> acq_function = AcqFunctionEI$new()

> acq_function = acqf("ei")

> acqf("cb")
<AcqFunctionCB:acq_cb>
Domain:
<ParamSet(0)>
Empty.
Codomain:
<Codomain(0)>
Empty.
Constants:
<ParamSet(1)>
       id    class lower upper nlevels default  value
   <char>   <char> <num> <num>   <num>  <list> <list>
1: lambda ParamDbl     0   Inf     Inf       2      2

> library(bbotk)
Loading required package: paradox

> acq_optimizer = AcqOptimizer$new(opt("random_search"), 
+     terminator = trm("evals"))

> acq_optimizer = acqo(opt("random_search"), terminator = trm("evals"))

> acq_optimizer$optimizer
<OptimizerBatchRandomSearch>: Random Search
* Parameters: batch_size=1
* Parameter classes: ParamLgl, ParamInt, ParamDbl, ParamFct
* Properties: dependencies, single-crit, multi-crit
* Packages: bbotk

> acq_optimizer$terminator
<TerminatorEvals>: Number of Evaluation
* Parameters: n_evals=100, k=0

> acq_optimizer$param_set
<ParamSet(6)>
Key: <id>
                       id    class lower upper nlevels        default   parents
                   <char>   <char> <num> <num>   <num>         <list>    <list>
1:           catch_errors ParamLgl    NA    NA       2           TRUE    [NULL]
2:          logging_level ParamFct    NA    NA       6           warn    [NULL]
3:           n_candidates ParamInt     1   Inf     Inf              1    [NULL]
4: skip_already_evaluated ParamLgl    NA    NA       2           TRUE    [NULL]
5:              warmstart ParamLgl    NA    NA       2          FALSE    [NULL]
6:         warmstart_size ParamInt     1   Inf     Inf <NoDefault[0]> warmstart
    value
   <list>
1:   TRUE
2:   warn
3:      1
4:   TRUE
5:  FALSE
6: [NULL]

> optimizer = OptimizerMbo$new(bayesopt_ego, surrogate = surrogate, 
+     acq_function = acq_function, acq_optimizer = acq_optimizer)

> optimizer = opt("mbo", loop_function = bayesopt_ego, 
+     surrogate = surrogate, acq_function = acq_function, acq_optimizer = acq_optimizer)

> optimizer
<OptimizerMbo>: Model Based Optimization
* Parameter classes: ParamLgl, ParamInt, ParamDbl
* Properties: single-crit
* Packages: mlr3mbo, mlr3, mlr3learners, DiceKriging, bbotk
* Loop function: bayesopt_ego
* Surrogate: LearnerRegrKM
* Acquisition Function: AcqFunctionEI
* Acquisition Function Optimizer: (OptimizerBatchRandomSearch |
  TerminatorEvals)

> as.data.table(mlr_result_assigners)
Key: <key>
         key                     label                                     man
      <char>                    <char>                                  <char>
1:   archive                   Archive   mlr3mbo::mlr_result_assigners_archive
2: surrogate Mean Surrogate Prediction mlr3mbo::mlr_result_assigners_surrogate

> result_assigner = ResultAssignerArchive$new()

> result_assigner = ras("archive")

> tuner = TunerMbo$new(bayesopt_ego, surrogate = surrogate, 
+     acq_function = acq_function, acq_optimizer = acq_optimizer)

> mlr3misc::get_private(tuner)[[".optimizer"]]
<OptimizerMbo>: Model Based Optimization
* Parameter classes: ParamLgl, ParamInt, ParamDbl
* Properties: single-crit
* Packages: mlr3mbo, mlr3, mlr3learners, DiceKriging, bbotk
* Loop function: bayesopt_ego
* Surrogate: LearnerRegrKM
* Acquisition Function: AcqFunctionEI
* Acquisition Function Optimizer: (OptimizerBatchRandomSearch |
  TerminatorEvals)

> set.seed(2906)

> domain = ps(x = p_dbl(lower = -1, upper = 1))

> codomain = ps(y = p_dbl(tags = "minimize"))

> objective_function = function(xs) {
+     list(y = xs$x^2)
+ }

> objective = ObjectiveRFun$new(fun = objective_function, 
+     domain = domain, codomain = codomain)

> instance = OptimInstanceBatchSingleCrit$new(objective = objective, 
+     terminator = trm("evals", n_evals = 10))

> initial_design = data.table(x = rep(1, 4))

> instance$eval_batch(initial_design)
INFO  [21:05:14.351] [bbotk] Evaluating 4 configuration(s)
INFO  [21:05:14.755] [bbotk] Result of batch 1:
INFO  [21:05:14.802] [bbotk]  x y
INFO  [21:05:14.802] [bbotk]  1 1
INFO  [21:05:14.802] [bbotk]  1 1
INFO  [21:05:14.802] [bbotk]  1 1
INFO  [21:05:14.802] [bbotk]  1 1

> surrogate = srlrn(default_gp())

> acq_function = acqf("ei")

> acq_optimizer = acqo(opt("random_search", batch_size = 1000), 
+     terminator = trm("evals", n_evals = 1000))

> optimizer = opt("mbo", loop_function = bayesopt_ego, 
+     surrogate = surrogate, acq_function = acq_function, acq_optimizer = acq_optimizer)

> optimizer$optimize(instance)
INFO  [21:05:15.700] [bbotk] Starting to optimize 1 parameter(s) with '<OptimizerMbo>' and '<TerminatorEvals> [n_evals=10, k=0]'
WARN  [21:05:16.084] [bbotk] missing value where TRUE/FALSE needed
INFO  [21:05:16.088] [bbotk] surrogate_update_error / mbo_error / error / condition
INFO  [21:05:16.092] [bbotk] Proposing a randomly sampled point
INFO  [21:05:16.170] [bbotk] Evaluating 1 configuration(s)
INFO  [21:05:16.224] [bbotk] Result of batch 2:
INFO  [21:05:16.231] [bbotk]          x         y
INFO  [21:05:16.231] [bbotk]  0.6742299 0.4545859
INFO  [21:05:17.004] [bbotk] Evaluating 1 configuration(s)
INFO  [21:05:17.051] [bbotk] Result of batch 3:
INFO  [21:05:17.060] [bbotk]          x  x_domain     acq_ei .already_evaluated         y
INFO  [21:05:17.060] [bbotk]  0.6686506 <list[1]> 0.04358144              FALSE 0.4470936
INFO  [21:05:17.911] [bbotk] Evaluating 1 configuration(s)
INFO  [21:05:17.955] [bbotk] Result of batch 4:
INFO  [21:05:17.963] [bbotk]          x  x_domain    acq_ei .already_evaluated         y
INFO  [21:05:17.963] [bbotk]  0.3883431 <list[1]> 0.1264502              FALSE 0.1508104
INFO  [21:05:18.653] [bbotk] Evaluating 1 configuration(s)
INFO  [21:05:18.698] [bbotk] Result of batch 5:
INFO  [21:05:18.706] [bbotk]           x  x_domain    acq_ei .already_evaluated         y
INFO  [21:05:18.706] [bbotk]  -0.5325387 <list[1]> 0.2231679              FALSE 0.2835975
INFO  [21:05:19.436] [bbotk] Evaluating 1 configuration(s)
INFO  [21:05:19.482] [bbotk] Result of batch 6:
INFO  [21:05:19.490] [bbotk]           x  x_domain    acq_ei .already_evaluated           y
INFO  [21:05:19.490] [bbotk]  0.04818857 <list[1]> 0.1256543              FALSE 0.002322138
INFO  [21:05:20.156] [bbotk] Evaluating 1 configuration(s)
INFO  [21:05:20.201] [bbotk] Result of batch 7:
INFO  [21:05:20.225] [bbotk]            x  x_domain      acq_ei .already_evaluated            y
INFO  [21:05:20.225] [bbotk]  -0.01951836 <list[1]> 0.002040261              FALSE 0.0003809664
INFO  [21:05:20.458] [bbotk] Finished optimizing after 10 evaluation(s)
INFO  [21:05:20.461] [bbotk] Result:
INFO  [21:05:20.467] [bbotk]            x  x_domain            y
INFO  [21:05:20.467] [bbotk]        <num>    <list>        <num>
INFO  [21:05:20.467] [bbotk]  -0.01951836 <list[1]> 0.0003809664
             x  x_domain            y
         <num>    <list>        <num>
1: -0.01951836 <list[1]> 0.0003809664

> instance$archive$data
              x            y  x_domain           timestamp batch_nr      acq_ei
          <num>        <num>    <list>              <POSc>    <int>       <num>
 1:  1.00000000 1.0000000000 <list[1]> 2024-09-15 21:05:14        1          NA
 2:  1.00000000 1.0000000000 <list[1]> 2024-09-15 21:05:14        1          NA
 3:  1.00000000 1.0000000000 <list[1]> 2024-09-15 21:05:14        1          NA
 4:  1.00000000 1.0000000000 <list[1]> 2024-09-15 21:05:14        1          NA
 5:  0.67422986 0.4545859064 <list[1]> 2024-09-15 21:05:16        2          NA
 6:  0.66865060 0.4470936288 <list[1]> 2024-09-15 21:05:17        3 0.043581438
 7:  0.38834314 0.1508103944 <list[1]> 2024-09-15 21:05:17        4 0.126450231
 8: -0.53253872 0.2835974922 <list[1]> 2024-09-15 21:05:18        5 0.223167948
 9:  0.04818857 0.0023221379 <list[1]> 2024-09-15 21:05:19        6 0.125654255
10: -0.01951836 0.0003809664 <list[1]> 2024-09-15 21:05:20        7 0.002040261
    .already_evaluated
                <lgcl>
 1:                 NA
 2:                 NA
 3:                 NA
 4:                 NA
 5:                 NA
 6:              FALSE
 7:              FALSE
 8:              FALSE
 9:              FALSE
10:              FALSE

> instance$archive$clear()

> instance$eval_batch(initial_design)
INFO  [21:05:20.486] [bbotk] Evaluating 4 configuration(s)
INFO  [21:05:20.603] [bbotk] Result of batch 1:
INFO  [21:05:20.609] [bbotk]  x y
INFO  [21:05:20.609] [bbotk]  1 1
INFO  [21:05:20.609] [bbotk]  1 1
INFO  [21:05:20.609] [bbotk]  1 1
INFO  [21:05:20.609] [bbotk]  1 1

> optimizer$surrogate$param_set$values$catch_errors = FALSE

> optimizer$optimize(instance)
INFO  [21:05:20.635] [bbotk] Starting to optimize 1 parameter(s) with '<OptimizerMbo>' and '<TerminatorEvals> [n_evals=10, k=0]'

  When sourcing ‘mlr3mbo.R’:
Error: missing value where TRUE/FALSE needed
Execution halted
be-marc commented 1 month ago

Can you test the current main branch? This error is maybe caused by the mlr3 dev version.

sumny commented 4 weeks ago

The vignette was dropped anyways in the latest changes (starting > 0.2.4), because it is superseded by the book chapter: https://mlr3book.mlr-org.com/chapters/chapter5/advanced_tuning_methods_and_black_box_optimization.html#sec-bayesian-optimization

So I am closing this now.