resample$score(predict_sets = 'test') gives same output as resample$score(predict_sets = 'train'). Also, is there any functionality where results from both train and test can be accessed?

mlr-org / mlr3

mlr3: Machine Learning in R - next generation

https://mlr3.mlr-org.com

GNU Lesser General Public License v3.0

936 stars 85 forks source link

resample$score(predict_sets = 'test') gives same output as resample$score(predict_sets = 'train'). Also, is there any functionality where results from both train and test can be accessed? #950

Closed arun-chhikara-ta closed 1 year ago

arun-chhikara-ta commented 1 year ago

Description

In both the situations, i think it just outputs test results ...

Reproducible example

tasks = tsks(c("german_credit", "sonar")) learners = lrns(c("classif.rpart", "classif.ranger","classif.featureless"), predict_type = "prob") rsmp_cv5 = rsmp("cv", folds = 5) set.seed(1234) design = benchmark_grid(tasks, learners, rsmp_cv5) bmr = benchmark(design) bmr$score(predict_sets = 'train') bmr$score(predict_sets = 'test')