mlr-org / mlr3

mlr3: Machine Learning in R - next generation
https://mlr3.mlr-org.com
GNU Lesser General Public License v3.0
927 stars 86 forks source link

Possibility to save / access models via callbacks during resample #899

Open sebffischer opened 1 year ago

sebffischer commented 1 year ago

In some cases, users might want to fit large models on large datasets using resample(). In this case here, the person was unable to fit the models into RAM when specifying store_models = TRUE, but he still wanted access to the models, or rather the impurity scores from the models.

I have to suggestions to make these things more flexible:

  1. Adding callbacks in resample() would allow to save these models to disk during resample().
  2. Can we somehow score some measures that depend on the model on the workers? So that measures that require the model can still be evaluated when store_models = FALSE?
be-marc commented 1 week ago

Same as #1013.