search

tidymodels / tune

Tools for tidy parameter tuning
https://tune.tidymodels.org
Other
280 stars 42 forks source link

Suggestion: Examples of using nested resampling for hyperparameter tuning #611

Open MxNl opened 1 year ago

MxNl commented 1 year ago

I recently really struggled to use nested resampling in combination with workflowsets and fitting them using workflow_map and fit_resamples for hyperparameter tuning. I could not get all the bits and pieces of streamlining this together, even after reading this article from the website. To me the implementation of nested resampling seemed a bit "hacky" compared to all the well streamlined workflows in tidymodels. Or it is due to my limited brain capacity :-). This is why I thought that it might be good to add some examples of "how to do nested resampling" with other workflows such as workflowsets. Anyhow, thanks for your great work and also generally super good documentation!

juliasilge commented 1 year ago

Thanks for this feedback @MxNl! 🙌 It is definitely not you 😄 but rather the limited ways in which nested resampling is currently supported.

I'm going to move this over to tune and link it to tidymodels/tune#214 where another person has asked for similar features. Then, this can be prioritized with other requests for new features. As one small initial step, it may be good to more clearly document that nested resampling is not currently supported in tune.

MxNl commented 1 year ago

Cool, support for nested resampling would be great! I am looking forward to it! Thanks for your quick reply :-)