search

tidymodels / stacks

An R package for tidy stacked ensemble modeling
https://stacks.tidymodels.org
Other
295 stars 27 forks source link

`multisession` failure when using extension packages #215

Closed simonpcouch closed 2 months ago

simonpcouch commented 5 months ago

When making patches to fix runs on general-meta, I noticed that multisession workers required that needed parsnip extension packages be loaded via options.future inside of fit_members():

https://github.com/tidymodels/stacks/blob/f2e33419b5712ae7df3ca4d689c652188dce4b59/R/fit_members.R#L157

I wonder if we could find some way to restore the current parsnip model environment parsnip::get_model_env() in each worker. I also wonder how/why we don't tend to see this with normal tune runs.

simonpcouch commented 5 months ago

Related-ish to #118.

simonpcouch commented 5 months ago

This is likely a matter of passing required_pkgs(workflow) to .options.future.

simonpcouch commented 2 months ago

This may actually be a model registration issue--see https://github.com/tidymodels/parsnip/issues/1128.

simonpcouch commented 2 months ago

Based on that reprex in parsnip, I'm going to go ahead and close—we shouldn't need to pass any future options there, instead relying on set_dependency() calls in the model registration. If we run into this again, check that first!