mlr-org / mlr3learners

Recommended learners for mlr3
https://mlr3learners.mlr-org.com
GNU Lesser General Public License v3.0
89 stars 14 forks source link

feat: add early stopping on test set #234

Closed be-marc closed 1 year ago

be-marc commented 2 years ago

The test sets are added to the watchlist and early stopping is performed.

learner = lrn("classif.xgboost", 
  early_stopping_rounds = 10,
  early_stopping_set  = "test", 
  nrounds = 100, 
  eval_metric = "error")

rr = resample(tsk("pima"), learner, rsmp("holdout"), store_models = TRUE)
rr$learners[[1]]$model$evaluation_log

#>     iter train_error test_error
#>  1:    1    0.144531   0.289062
#>  2:    2    0.119141   0.234375
#>  3:    3    0.117188   0.230469
#>  4:    4    0.099609   0.250000
#>  5:    5    0.089844   0.246094
#>  6:    6    0.083984   0.230469
#>  7:    7    0.074219   0.238281
#>  8:    8    0.064453   0.242188
#>  9:    9    0.060547   0.250000
#> 10:   10    0.046875   0.246094
#> 11:   11    0.044922   0.253906
#> 12:   12    0.041016   0.250000
#> 13:   13    0.035156   0.242188
be-marc commented 2 years ago

Replaced by #235

be-marc commented 2 years ago

Merge with https://github.com/mlr-org/mlr3/pull/853

codecov-commenter commented 2 years ago

Codecov Report

Merging #234 (735c517) into main (f78c463) will decrease coverage by 0.10%. The diff coverage is 92.30%.

:exclamation: Current head 735c517 differs from pull request most recent head 703f715. Consider uploading reports for the commit 703f715 to get more accurate results

@@            Coverage Diff             @@
##             main     #234      +/-   ##
==========================================
- Coverage   96.90%   96.80%   -0.11%     
==========================================
  Files          25       25              
  Lines        1356     1376      +20     
==========================================
+ Hits         1314     1332      +18     
- Misses         42       44       +2     
Impacted Files Coverage Δ
R/LearnerClassifXgboost.R 98.17% <92.30%> (-0.54%) :arrow_down:
R/LearnerRegrXgboost.R 98.47% <92.30%> (-0.71%) :arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

be-marc commented 1 year ago

@mllg Please (review and) merge