mlr-org / bbotk

Black-box optimization framework for R.
https://bbotk.mlr-org.com
GNU Lesser General Public License v3.0
21 stars 9 forks source link

Log should print how many evals the terminator is using #102

Closed pat-s closed 3 years ago

pat-s commented 4 years ago

Currently only #> INFO [21:26:05.492] Starting to optimize 2 parameter(s) with '<OptimizerRandomSearch>' and '<TerminatorEvals>' is shown.



    library(mlr3)
    library(mlr3tuning)
    library(paradox)
    library(mlr3learners)

    search_space = ParamSet$new(params = list(
      ParamInt$new("k", lower = 1, upper = 50),
      ParamDbl$new("distance", lower = 1, upper = 50)
    ))

    at <- AutoTuner$new(
      learner = lrn("classif.kknn"),
      resampling = rsmp("cv", folds = 5),
      measure = msr("classif.ce"),
      search_space = search_space,
      #tune_ps = search_space,
      terminator = trm("evals", n_evals = 3),
      tuner = tnr("random_search")
    )

    design <- benchmark_grid(
      tasks = tsk("iris"), learners = list(at),
      resamplings = rsmp("cv", folds = 5)
    )
    bm <- benchmark(design)
    #> INFO  [21:26:05.412] Result of batch 3: 
    #> INFO  [21:26:05.413]   k distance classif.ce      resample_result 
    #> INFO  [21:26:05.413]  10 26.28679       0.05 <ResampleResult[18]> 
    #> INFO  [21:26:05.426] Finished optimizing after 3 evaluation(s) 
    #> INFO  [21:26:05.429] Result: 
    #> INFO  [21:26:05.432]   k distance learner_param_vals  x_domain classif.ce 
    #> INFO  [21:26:05.432]  10 26.28679          <list[2]> <list[2]>       0.05 
    #> INFO  [21:26:05.475] Applying learner 'classif.kknn.tuned' on task 'iris' (iter 4/5) 
    #> Classes 'TuningInstanceSingleCrit', 'OptimInstanceSingleCrit', 'OptimInstance', 'R6' <TuningInstanceSingleCrit> 
    #> INFO  [21:26:05.492] Starting to optimize 2 parameter(s) with '<OptimizerRandomSearch>' and '<TerminatorEvals>' 
   [...]

<sup>Created on 2020-08-08 by the [reprex package](https://reprex.tidyverse.org) (v0.3.0.9001)</sup>
jakob-r commented 3 years ago

We will put basic params in $format method so its always printed.

Also, we have progress bars now/soon.

be-marc commented 3 years ago

Logger looks like this now.

Starting to optimize 2 parameter(s) with '<OptimizerGenSA>' and '<TerminatorEvals> [n_evals=20]' 

or

Starting to optimize 2 parameter(s) with '<OptimizerGenSA>' and '<TerminatorStagnation> [iters=10, threshold=0.1]'