Feature Request: additional parallel ability

[lg1000]

Hi,

it's a shame, that github doesn't offer a feature request button, so I need to ask here.

Is there any planning on adding further parallel abilities to the framework? I am asking, because modeltime_forecast() and modeltime_calibrate() take about 2 minutes for my less than 200 time series (global ensemble model, xgb and elnet) on my MacBookPro. This is my biggest overhead right now and I have no idea how to speed those functions up, without the ability to process in parallel.

Regards!

[mdancho84]

I've been thinking about this one too. Calibration and forecasting can be expensive too and should be able to be parallelized using the same patterns we have implemented in the fitting functions. It may take a bit to get to this.

[lg1000]

Thanks for the heads up! I am looking forward to this feature, as some costumers need to use their private Laptop for my forecast application sometime in home office. Two minutes versus 4 is a big improvement in this case. However there needs to be a GPU interface as well to take it all to another level. Is there any possibility to leverage GPU as well with modeltime?

[mdancho84]

I've reopened the issue to keep it for tracking. The main actions are:

• [ ] Parallel Calibration: modeltime_calibrate()
• [ ] Parallel Forecasting: modeltime_forecast()

[mdancho84]

GPU support is dependent on the algorithm and your machine.

• XGBoost can be run on GPU if properly set up. https://github.com/dmlc/xgboost/blob/master/R-package/demo/gpu_accelerated.R

AND something to consider is how GPU vs CPU will run during parallel processing.

[lg1000]

Sounds good! Can't wait to experiment, tuning some models with my new NVIDIA GPU

[AlbertoAlmuinha]

Hi @mdancho84 ,

I've modified both modeltime_calibrate() and modeltime_forecast() to allow parallel support following exactly the same principles as in modeltime_refit() (code is very similar in style).

In modeltime_forecast I have parallelized both safe_modeltime_forecast_map and safe_conf_interval_map functions (both by id and without id).

I will make a new PR as soon as the one that is currently active is closed so as not to mix topics in the same PR.

We discuss whatever.

Regards,

[lg1000]

Hi @AlbertoAlmuinha,

thanks for the update! If I understand you correctly, I am now able to use modeltime_calibrate and modeltime_forecast inside the functions "parallel_start" and "parallel_stop". Is this correct?

[AlbertoAlmuinha]

Not yet @lg1000 . I need to make a PR with the changes in the next days,but right now there is another PR going on and I don't want to mix topics

[lg1000]

Alright, looking forward to it!

[jaredlander]

I've reopened the issue to keep it for tracking. The main actions are:

• [ ] Parallel Calibration: modeltime_calibrate()
• [ ] Parallel Forecasting: modeltime_forecast()

I think it's also an issue during the regular fitting process. The parsnip::fit() part of the code below runs in parallel.

arima_rec <- recipes::recipe(n ~ day, rsample::training(splits)) |> 
    timetk::step_timeseries_signature(day) |> 
    recipes::step_select(n, day, day_wday.lbl, day_month.lbl)

arima_spec <- modeltime::arima_reg() |> 
    parsnip::set_engine(engine = "auto_arima")

arima_day_month_spec_wf <- workflows::workflow(arima_rec, arima_spec)

arima_day_month_fitted <- parsnip::fit(
    arima_day_month_spec_wf, 
    rsample::training(splits)
)