search

business-science / modeltime

Modeltime unlocks time series forecast models and machine learning in one framework
https://business-science.github.io/modeltime/
Other
535 stars 82 forks source link

Is tuning parameters with recursive forecasting possible? #60

Closed tonyk7440 closed 3 years ago

tonyk7440 commented 3 years ago

Hi All, I've been doing some forecasting using modeltime but have ran into some scaling issues when extending out the horizon as features such as lags < horizon cannot then be used and a new model is needed. I've seen the enhancement with recursive forecasting which solves this but I'm wondering if it is possible to tune parameters though tune_grid and use the recursive forecasting together?

mdancho84 commented 3 years ago

This is a good question. I haven't tried (or tested) recursive() ability to work with tune::tune_grid(). I will keep this issue open as I have time to address. We are also looking at solutions for panel data #67, and maybe this aligns with that issue.

mdancho84 commented 3 years ago

Circling back on this. The Short Answer - No.

The Long Answer

We've been tuning the algorithms without recursive because we have access to the lags. Is this perfect - No. But the way we set up recursive() is to modify the fitted model.

I'll need to investigate whether or not we can apply recursive internally to fit_resamples() and tune_grid() (and friends). It may be possible to incorporate into the process if we can access the fitted models that are created during this process, but this will take some investigation.

So for now, I recommend tuning the models with the lags that are already available in the testing sets, and eventually we can assess whether or not adding recursive to the training process is possible.

mdancho84 commented 3 years ago

Closing this issue.