search

robjhyndman / forecast

Forecasting Functions for Time Series and Linear Models
http://pkg.robjhyndman.com/forecast
1.1k stars 340 forks source link

Parallel runs of auto.arima unnecessarily slow - parLapply vs. mclapply #889

Closed jonlachmann closed 2 years ago

jonlachmann commented 2 years ago

Hi! I am using the auto.arima function from the forecast package for a project where we estimate a vast number of different models, some are large and some are small. The issue I am having is that I want to run auto.arima parallelized, but for many models this results in a longer runtime than running it single core.

At the moment the clustered R parallel function parLapply is used, which is probably a good idea when running on a cluster and evaluating a huge model. However, when running on a single machine, and a lot of the time with smaller models, an awful lot of time is spent setting up and destroying the cluster, just to perform calculations that would have been faster to just run sequentially.

What I would want is to have an option to use mclapply instead, allowing me to maximize performance for all (non-tiny) models. If you want to I could fork the repository and create a pull requested with some suggested changes. Is there a firm reason to use parLapply at all?

Best regards and thank you for a very useful package! =)

mitchelloharawild commented 2 years ago

Resolved by #891

mitchelloharawild commented 2 years ago

Thanks for your PR!