search

alkaline-ml / pmdarima

A statistical library designed to fill the void in Python's time series analysis capabilities, including the equivalent of R's auto.arima function.
https://www.alkaline-ml.com/pmdarima
MIT License
1.57k stars 231 forks source link

Force Stepwise to Explore All Possibilities #438

Closed AlexanderLavelle closed 3 years ago

AlexanderLavelle commented 3 years ago

Hello - My boss uses R but my peer group uses Python. We are doing ts forecasting and using pmdarima for AR/MA ordering.

Is there a way to force the stepwise search to exhaust available options before finishing? It appears to use some sort of gradient but sometimes does not persist to the higher orders with lower AIC.

I also utilize StepwiseContext(max_iterations=100), max_order >= 20, and max_iter=100, but the search rarely utilizes the entire grid and may not force the best models.

This behavior apparently is different than R, where I am told the behavior exhausts the grid.

Do you have any recommendations to expand the search OR to behave as in R?

Thank you in advance!

System: python: 3.8.5 (default, Sep 3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] executable: C:\ProgramFiles\Anaconda3\python.exe machine: Windows-10-10.0.18362-SP0

Python dependencies: pip: 21.0.1 setuptools: 52.0.0.post20210125 sklearn: 0.24.2 statsmodels: 0.12.2 numpy: 1.20.1 scipy: 1.6.2 Cython: 0.29.23 pandas: 1.2.4 joblib: 1.0.1 pmdarima: 1.8.2 Windows-10-10.0.18362-SP0 Python 3.8.5 (default, Sep 3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] pmdarima 1.8.2 NumPy 1.20.1 SciPy 1.6.2 Scikit-Learn 0.24.2 Statsmodels 0.12.2

tgsmith61591 commented 3 years ago

Hi Alexander,

There is a misconception here -- the stepwise algorithm is not exhaustive (in either R or pmdarima). It follows the step-wise algorithm laid out in section 2.2 of Rob Hyndman's 2008 paper: "Automatic Time Series Forecasting".

If you want to perform a sweep of the entire parameter space (honoring max_p, max_q, max_P, max_Q and max_order), you can achieve this without the stepwise algorithm, but it's generally slower:

In [1]: import pmdarima as pm

In [2]: y = pm.datasets.load_wineind()

In [3]: model = pm.auto_arima(y, stepwise=False, trace=3)
 ARIMA(0,1,0)(0,0,0)[1] intercept   : AIC=3587.776, Time=0.03 sec
 ARIMA(0,1,1)(0,0,0)[1] intercept   : AIC=3509.903, Time=0.07 sec
 ARIMA(0,1,2)(0,0,0)[1] intercept   : AIC=3506.259, Time=0.12 sec
 ARIMA(0,1,3)(0,0,0)[1] intercept   : AIC=3506.173, Time=0.10 sec
 ARIMA(0,1,4)(0,0,0)[1] intercept   : AIC=3503.947, Time=0.16 sec
 ARIMA(0,1,5)(0,0,0)[1] intercept   : AIC=3495.968, Time=0.24 sec
 ARIMA(1,1,0)(0,0,0)[1] intercept   : AIC=3573.940, Time=0.01 sec
 ARIMA(1,1,1)(0,0,0)[1] intercept   : AIC=3508.854, Time=0.14 sec
 ARIMA(1,1,2)(0,0,0)[1] intercept   : AIC=3505.978, Time=0.17 sec
Near non-invertible roots for order (1, 1, 3)(0, 0, 0, 1); setting score to inf (at least one inverse root too close to the border of the unit circle: 1.000)
 ARIMA(1,1,3)(0,0,0)[1] intercept   : AIC=inf, Time=0.23 sec
 ARIMA(1,1,4)(0,0,0)[1] intercept   : AIC=3504.179, Time=0.26 sec
 ARIMA(2,1,0)(0,0,0)[1] intercept   : AIC=3540.400, Time=0.03 sec
 ARIMA(2,1,1)(0,0,0)[1] intercept   : AIC=3500.610, Time=0.10 sec
 ARIMA(2,1,2)(0,0,0)[1] intercept   : AIC=3503.442, Time=0.16 sec
Near non-invertible roots for order (2, 1, 3)(0, 0, 0, 1); setting score to inf (at least one inverse root too close to the border of the unit circle: 0.996)
 ARIMA(2,1,3)(0,0,0)[1] intercept   : AIC=inf, Time=0.28 sec
 ARIMA(3,1,0)(0,0,0)[1] intercept   : AIC=3516.118, Time=0.03 sec
 ARIMA(3,1,1)(0,0,0)[1] intercept   : AIC=3502.598, Time=0.15 sec
 ARIMA(3,1,2)(0,0,0)[1] intercept   : AIC=3504.888, Time=0.08 sec
 ARIMA(4,1,0)(0,0,0)[1] intercept   : AIC=3516.442, Time=0.03 sec
 ARIMA(4,1,1)(0,0,0)[1] intercept   : AIC=3501.754, Time=0.19 sec
 ARIMA(5,1,0)(0,0,0)[1] intercept   : AIC=3517.051, Time=0.04 sec

Best model:  ARIMA(0,1,5)(0,0,0)[1] intercept
Total fit time: 2.634 seconds

And one with seasonality:

In [4]: model = pm.auto_arima(y, stepwise=False, trace=3, seasonal=True, m=12)
 ARIMA(0,1,0)(0,1,0)[12]             : AIC=3131.408, Time=0.02 sec
 ARIMA(0,1,0)(0,1,1)[12]             : AIC=3117.921, Time=0.15 sec
 ARIMA(0,1,0)(0,1,2)[12]             : AIC=3116.480, Time=0.41 sec
 ARIMA(0,1,0)(1,1,0)[12]             : AIC=3122.194, Time=0.05 sec
 ARIMA(0,1,0)(1,1,1)[12]             : AIC=3115.814, Time=0.22 sec
 ARIMA(0,1,0)(1,1,2)[12]             : AIC=3117.814, Time=1.15 sec
 ARIMA(0,1,0)(2,1,0)[12]             : AIC=3119.870, Time=0.13 sec
 ARIMA(0,1,0)(2,1,1)[12]             : AIC=3119.499, Time=0.38 sec
Near non-invertible roots for order (0, 1, 0)(2, 1, 2, 12); setting score to inf (at least one inverse root too close to the border of the unit circle: 1.000)
 ARIMA(0,1,0)(2,1,2)[12]             : AIC=inf, Time=1.24 sec
 ARIMA(0,1,1)(0,1,0)[12]             : AIC=3089.456, Time=0.03 sec
 ARIMA(0,1,1)(0,1,1)[12]             : AIC=3066.329, Time=0.08 sec
 ARIMA(0,1,1)(0,1,2)[12]             : AIC=3067.481, Time=0.24 sec
 ARIMA(0,1,1)(1,1,0)[12]             : AIC=3071.631, Time=0.08 sec
 ARIMA(0,1,1)(1,1,1)[12]             : AIC=3067.457, Time=0.15 sec
Near non-invertible roots for order (0, 1, 1)(1, 1, 2, 12); setting score to inf (at least one inverse root too close to the border of the unit circle: 1.000)
 ARIMA(0,1,1)(1,1,2)[12]             : AIC=inf, Time=1.49 sec
 ARIMA(0,1,1)(2,1,0)[12]             : AIC=3068.924, Time=0.19 sec
 ARIMA(0,1,1)(2,1,1)[12]             : AIC=3069.456, Time=0.49 sec
Near non-invertible roots for order (0, 1, 1)(2, 1, 2, 12); setting score to inf (at least one inverse root too close to the border of the unit circle: 0.997)
 ARIMA(0,1,1)(2,1,2)[12]             : AIC=inf, Time=1.10 sec
 ARIMA(0,1,2)(0,1,0)[12]             : AIC=3087.883, Time=0.04 sec
 ARIMA(0,1,2)(0,1,1)[12]             : AIC=3065.533, Time=0.10 sec
 ARIMA(0,1,2)(0,1,2)[12]             : AIC=3066.373, Time=0.29 sec
 ARIMA(0,1,2)(1,1,0)[12]             : AIC=3070.728, Time=0.09 sec
 ARIMA(0,1,2)(1,1,1)[12]             : AIC=3066.239, Time=0.18 sec
Near non-invertible roots for order (0, 1, 2)(1, 1, 2, 12); setting score to inf (at least one inverse root too close to the border of the unit circle: 0.998)
 ARIMA(0,1,2)(1,1,2)[12]             : AIC=inf, Time=0.97 sec
 ARIMA(0,1,2)(2,1,0)[12]             : AIC=3068.279, Time=0.25 sec
 ARIMA(0,1,2)(2,1,1)[12]             : AIC=3068.215, Time=0.48 sec
 ARIMA(0,1,3)(0,1,0)[12]             : AIC=3090.078, Time=0.09 sec
 ARIMA(0,1,3)(0,1,1)[12]             : AIC=3066.351, Time=0.20 sec
 ARIMA(0,1,3)(0,1,2)[12]             : AIC=3067.383, Time=0.44 sec
 ARIMA(0,1,3)(1,1,0)[12]             : AIC=3071.767, Time=0.16 sec
 ARIMA(0,1,3)(1,1,1)[12]             : AIC=3067.277, Time=0.26 sec
 ARIMA(0,1,3)(2,1,0)[12]             : AIC=3069.412, Time=0.44 sec
 ARIMA(0,1,4)(0,1,0)[12]             : AIC=3092.299, Time=0.11 sec
 ARIMA(0,1,4)(0,1,1)[12]             : AIC=3068.314, Time=0.23 sec
 ARIMA(0,1,4)(1,1,0)[12]             : AIC=3073.854, Time=0.18 sec
 ARIMA(0,1,5)(0,1,0)[12]             : AIC=3093.416, Time=0.13 sec
 ARIMA(1,1,0)(0,1,0)[12]             : AIC=3110.398, Time=0.03 sec
 ARIMA(1,1,0)(0,1,1)[12]             : AIC=3091.911, Time=0.18 sec
 ARIMA(1,1,0)(0,1,2)[12]             : AIC=3092.610, Time=0.19 sec
 ARIMA(1,1,0)(1,1,0)[12]             : AIC=3097.884, Time=0.07 sec
 ARIMA(1,1,0)(1,1,1)[12]             : AIC=3092.651, Time=0.12 sec
Near non-invertible roots for order (1, 1, 0)(1, 1, 2, 12); setting score to inf (at least one inverse root too close to the border of the unit circle: 0.999)
 ARIMA(1,1,0)(1,1,2)[12]             : AIC=inf, Time=0.92 sec
 ARIMA(1,1,0)(2,1,0)[12]             : AIC=3094.033, Time=0.17 sec
 ARIMA(1,1,0)(2,1,1)[12]             : AIC=3094.642, Time=0.53 sec
Near non-invertible roots for order (1, 1, 0)(2, 1, 2, 12); setting score to inf (at least one inverse root too close to the border of the unit circle: 0.997)
 ARIMA(1,1,0)(2,1,2)[12]             : AIC=inf, Time=0.78 sec
 ARIMA(1,1,1)(0,1,0)[12]             : AIC=3087.460, Time=0.08 sec
 ARIMA(1,1,1)(0,1,1)[12]             : AIC=3066.492, Time=0.20 sec
 ARIMA(1,1,1)(0,1,2)[12]             : AIC=3067.430, Time=0.49 sec
 ARIMA(1,1,1)(1,1,0)[12]             : AIC=3071.441, Time=0.18 sec
 ARIMA(1,1,1)(1,1,1)[12]             : AIC=3067.331, Time=0.23 sec
Near non-invertible roots for order (1, 1, 1)(1, 1, 2, 12); setting score to inf (at least one inverse root too close to the border of the unit circle: 0.997)
 ARIMA(1,1,1)(1,1,2)[12]             : AIC=inf, Time=1.16 sec
 ARIMA(1,1,1)(2,1,0)[12]             : AIC=3069.111, Time=0.37 sec
 ARIMA(1,1,1)(2,1,1)[12]             : AIC=3069.316, Time=0.66 sec
 ARIMA(1,1,2)(0,1,0)[12]             : AIC=3090.324, Time=0.08 sec
 ARIMA(1,1,2)(0,1,1)[12]             : AIC=3066.424, Time=0.22 sec
 ARIMA(1,1,2)(0,1,2)[12]             : AIC=3067.429, Time=0.54 sec
 ARIMA(1,1,2)(1,1,0)[12]             : AIC=3072.021, Time=0.22 sec
 ARIMA(1,1,2)(1,1,1)[12]             : AIC=3067.354, Time=0.34 sec
 ARIMA(1,1,2)(2,1,0)[12]             : AIC=3069.369, Time=0.69 sec
 ARIMA(1,1,3)(0,1,0)[12]             : AIC=3092.145, Time=0.17 sec
 ARIMA(1,1,3)(0,1,1)[12]             : AIC=3068.295, Time=0.41 sec
 ARIMA(1,1,3)(1,1,0)[12]             : AIC=3073.783, Time=0.17 sec
 ARIMA(1,1,4)(0,1,0)[12]             : AIC=3093.099, Time=0.25 sec
 ARIMA(2,1,0)(0,1,0)[12]             : AIC=3097.675, Time=0.04 sec
 ARIMA(2,1,0)(0,1,1)[12]             : AIC=3073.964, Time=0.09 sec
 ARIMA(2,1,0)(0,1,2)[12]             : AIC=3074.721, Time=0.28 sec
 ARIMA(2,1,0)(1,1,0)[12]             : AIC=3080.251, Time=0.10 sec
 ARIMA(2,1,0)(1,1,1)[12]             : AIC=3074.781, Time=0.18 sec
Near non-invertible roots for order (2, 1, 0)(1, 1, 2, 12); setting score to inf (at least one inverse root too close to the border of the unit circle: 0.997)
 ARIMA(2,1,0)(1,1,2)[12]             : AIC=inf, Time=0.79 sec
 ARIMA(2,1,0)(2,1,0)[12]             : AIC=3076.526, Time=0.23 sec
 ARIMA(2,1,0)(2,1,1)[12]             : AIC=3076.736, Time=0.53 sec
 ARIMA(2,1,1)(0,1,0)[12]             : AIC=3088.737, Time=0.06 sec
 ARIMA(2,1,1)(0,1,1)[12]             : AIC=3066.930, Time=0.17 sec
 ARIMA(2,1,1)(0,1,2)[12]             : AIC=3067.986, Time=0.45 sec
 ARIMA(2,1,1)(1,1,0)[12]             : AIC=3071.884, Time=0.16 sec
 ARIMA(2,1,1)(1,1,1)[12]             : AIC=3067.860, Time=0.31 sec
 ARIMA(2,1,1)(2,1,0)[12]             : AIC=3069.906, Time=0.44 sec
 ARIMA(2,1,2)(0,1,0)[12]             : AIC=3092.156, Time=0.14 sec
 ARIMA(2,1,2)(0,1,1)[12]             : AIC=3068.312, Time=0.34 sec
 ARIMA(2,1,2)(1,1,0)[12]             : AIC=3073.838, Time=0.36 sec
 ARIMA(2,1,3)(0,1,0)[12]             : AIC=3093.916, Time=0.28 sec
 ARIMA(3,1,0)(0,1,0)[12]             : AIC=3093.009, Time=0.05 sec
 ARIMA(3,1,0)(0,1,1)[12]             : AIC=3070.775, Time=0.13 sec
 ARIMA(3,1,0)(0,1,2)[12]             : AIC=3071.483, Time=0.34 sec
 ARIMA(3,1,0)(1,1,0)[12]             : AIC=3076.091, Time=0.12 sec
 ARIMA(3,1,0)(1,1,1)[12]             : AIC=3071.278, Time=0.22 sec
 ARIMA(3,1,0)(2,1,0)[12]             : AIC=3073.876, Time=0.28 sec
 ARIMA(3,1,1)(0,1,0)[12]             : AIC=3090.708, Time=0.08 sec
 ARIMA(3,1,1)(0,1,1)[12]             : AIC=3068.839, Time=0.22 sec
 ARIMA(3,1,1)(1,1,0)[12]             : AIC=3073.878, Time=0.22 sec
 ARIMA(3,1,2)(0,1,0)[12]             : AIC=3094.433, Time=0.09 sec
 ARIMA(4,1,0)(0,1,0)[12]             : AIC=3093.662, Time=0.06 sec
 ARIMA(4,1,0)(0,1,1)[12]             : AIC=3069.591, Time=0.14 sec
 ARIMA(4,1,0)(1,1,0)[12]             : AIC=3074.940, Time=0.16 sec
 ARIMA(4,1,1)(0,1,0)[12]             : AIC=3092.189, Time=0.20 sec
 ARIMA(5,1,0)(0,1,0)[12]             : AIC=3095.220, Time=0.07 sec

Best model:  ARIMA(0,1,2)(0,1,1)[12]
Total fit time: 29.106 seconds

Note that with stepwise=True we get the same result in significantly less time:

Best model:  ARIMA(0,1,2)(0,1,1)[12]
Total fit time: 5.059 seconds

For more info, take a look at the documentation for the auto_arima method.