mb706 / autoxgboost3

autoxgboost emulation for mlr3
Other
4 stars 0 forks source link

entry for booster in param set missing #2

Closed SebGGruber closed 4 years ago

SebGGruber commented 4 years ago

use task from mattermost file

axgb_settings = autoxgboost_space(task, tune.threshold = FALSE)
rsmp_inner = axgb_settings$resampling
learner = axgb_settings$learner
ps = axgb_settings$searchspace

# i added this and then it suddenly worked
ps$add(learner$param_set$params$xgboost.booster)
###########

ti = TuningInstance$new(task = task, learner = learner, resampling = rsmp_inner, param_set = ps, measures = msr("classif.ce"), terminator = term("evals", n_evals = 1))
tuner = tnr("random_search")
# here would be the error
tuner$tune(ti)
SebGGruber commented 4 years ago

More detailed explanation of the error:

I installed the newest github version of mlr3, mlr3tuning, mlr3pipelines, paradox, checkmates and autoxgboost and executed the following script:

library(autoxgboost)
library(mlr3tuning)
library(mlr3)
library(paradox)
library(checkmate)
data="data/abalone"
task = readRDS(file.path(data, "task.rds"))
axgb_settings = autoxgboost_space(task, tune.threshold = FALSE)
rsmp_inner = axgb_settings$resampling
learner = axgb_settings$learner
ps = axgb_settings$searchspace
ti = TuningInstance$new(task = task, learner = learner, resampling = rsmp_inner, param_set = ps, measures = msr("classif.ce"), terminator = term("evals", n_evals = 1))
tuner = tnr("random_search")
tuner$tune(ti)

which gives the following console output in a newly started R session:

> library(autoxgboost)
Loading required package: paradox
Loading required package: mlr3
Loading required package: mlr3pipelines
Loading required package: mlr3learners
> library(mlr3tuning)

Attaching package: ‘mlr3tuning’

The following object is masked from ‘package:stats’:

    terms

> library(mlr3)
> library(paradox)
> library(checkmate)
> data="data/abalone"
> task = readRDS(file.path(data, "task.rds"))
> axgb_settings = autoxgboost_space(task, tune.threshold = FALSE)
rsmp_inner = axgb_settings$resampling
> rsmp_inner = axgb_settings$resampling
> learner = axgb_settings$learner
> ps = axgb_settings$searchspace
> ti = TuningInstance$new(task = task, learner = learner, resampling = rsmp_inner, param_set = ps, measures = msr("classif.ce"), te
rminator = term("evals", n_evals = 1))
> tuner = tnr("random_search")
> tuner$tune(ti)
INFO  [13:22:06.027] Starting to tune 8 parameters with '<TunerRandomSearch>' and '<TerminatorEvals>'
INFO  [13:22:06.056] Terminator settings: n_evals=1
INFO  [13:22:06.108] Evaluating 1 configurations
INFO  [13:22:06.120]  xgboost.eta xgboost.gamma xgboost.max_depth xgboost.colsample_bytree xgboost.colsample_bylevel xgboost.lambda
 xgboost.alpha
INFO  [13:22:06.120]   0.08478911      5.979535                 5                 0.872689                 0.6943021      -8.741068
     -3.629845
INFO  [13:22:06.120]  xgboost.subsample
INFO  [13:22:06.120]          0.8994385
Error in (function (xs)  :
  Assertion on 'xs' failed: The parameter 'lambda' can only be set if the following condition is met 'booster = gblinear'. Instead
the parameter value for 'booster' is not set at all. Try setting 'booster' to a value that satisfies the condition.
>

R session info:

> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 19.10

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.7.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=de_DE.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=de_DE.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=de_DE.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] checkmate_2.0.0          mlr3tuning_0.1.2-9000    autoxgboost_0.0.0.9000   mlr3learners_0.1.6       mlr3pipelines_0.1.2.9000
[6] mlr3_0.1.7-9000          paradox_0.1.0-9000       nvimcom_0.9-83

loaded via a namespace (and not attached):
 [1] lgr_0.3.3           mlr3misc_0.1.8-9000 digest_0.6.25       crayon_1.3.4        withr_2.1.2         R6_2.4.1
 [7] mlr3measures_0.1.2  backports_1.1.5     uuid_0.1-4          data.table_1.12.8   tools_3.6.1         compiler_3.6.1
SebGGruber commented 4 years ago

forgot to add for clearance: data="data/abalone" should be the folder where the task.rds file I sent on mattermast is located on your PC

mb706 commented 4 years ago

Could you post the result of

sapply(c("mlr3", "mlr3pipelines", "mlr3tuning", "paradox", "mlr3misc"), remotes:::local_sha)

please?

mb706 commented 4 years ago

Ok I believe this was fixed in https://github.com/mlr-org/mlr3learners/commit/b6621a2e54ff8c98339a03f3bc38e11de4350a7d and updating mlr3learners should solve this.

SebGGruber commented 4 years ago

updated mlr3learners to github version and got a new error....

> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 19.10

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.7.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=de_DE.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=de_DE.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=de_DE.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] checkmate_2.0.0          mlr3tuning_0.1.2-9000
[3] autoxgboost_0.0.0.9000   mlr3learners_0.1.6-9000
[5] mlr3pipelines_0.1.2.9000 mlr3_0.1.8-9000
[7] paradox_0.1.0-9000       nvimcom_0.9-83

loaded via a namespace (and not attached):
 [1] lgr_0.3.3           mlr3misc_0.1.8-9000 digest_0.6.25
 [4] crayon_1.3.4        withr_2.1.2         R6_2.4.1
 [7] backports_1.1.5     uuid_0.1-4          data.table_1.12.8
[10] tools_3.6.1         compiler_3.6.1
> NvimR.selection(echo=TRUE)
> library(autoxgboost)
> library(mlr3tuning)
> library(checkmate)
> data="data/abalone"
> task = readRDS(file.path(data, "task.rds"))
> axgb_settings = autoxgboost_space(task, tune.threshold = FALSE)
> rsmp_inner = axgb_settings$resampling
> learner = axgb_settings$learner
> ps = axgb_settings$searchspace
> ti = TuningInstance$new(task = task, learner = learner, resampling = rsmp_
inner, param_set = ps, measures = msr("classif.ce"), terminator = term("ev .
.." ... [TRUNCATED]
> tuner = tnr("random_search")
> tuner$tune(ti)
INFO  [09:46:39.578] Starting to tune 8 parameters with '<TunerRandomSearch>
' and '<TerminatorEvals>'
INFO  [09:46:39.602] Terminator settings: n_evals=1
INFO  [09:46:39.664] Evaluating 1 configurations
INFO  [09:46:39.671]  xgboost.eta xgboost.gamma xgboost.max_depth xgboost.co
lsample_bytree
INFO  [09:46:39.671]     0.041469     -2.956749                17
     0.5267185
INFO  [09:46:39.671]  xgboost.colsample_bylevel xgboost.lambda xgboost.alpha
 xgboost.subsample
INFO  [09:46:39.671]                  0.9839101      -1.332195      0.759346
         0.9372812
INFO  [09:46:40.409] Benchmark with 1 resampling iterations
INFO  [09:46:40.414] Applying learner 'fixfactors.imputesample.encode.encode
impact.removeconstants.xgboost' on task 'abalone' (iter 1/1)
Error in FUN(X[[i]], ...) :
  unused argument (type = typeof(self$row_roles$use))
Error in FUN(X[[i]], ...) :
  unused argument (type = typeof(self$row_roles$use))
SebGGruber commented 4 years ago

here my package hashes:

                                      mlr3
"d9a85dc57bf3c5f45342192dee7e45ac2e79a6e3"
                             mlr3pipelines
"35174afea2b8b476a5fc91692a69f9402b4e1185"
                                mlr3tuning
"ccdbfcdd961b45c343b4b184915875a1885d6409"
                                   paradox
"15b60ad6d14102768683966d2cd672944c13fc27"
                                  mlr3misc
"f85c9f42e3dfa16e35d0e3bfff8ada4dd6508be8"
                              mlr3learners
"6f5088ad63b1e8361ff922f538bcae51b7679594"
                                 checkmate
"34a37a31064048ad747553628b26f621e5519e16"
                               autoxgboost
"41f3523582792fd332be1a2340ff6e24bfd2562d"
mb706 commented 4 years ago

this is probably because the task was generated by an old mlr3 version

mb706 commented 4 years ago

can you re-generate the task and check?

SebGGruber commented 4 years ago

i will try, but that's strange because then it also shouldn't work on your side, right?

mb706 commented 4 years ago

it didn't, lol. I always did something like

task = readRDS("task.rds")
task = TaskClassif$new("task", task$data(), task$target_names)
SebGGruber commented 4 years ago

thanks, it worked now!