rapidsai / cuml

cuML - RAPIDS Machine Learning Library
https://docs.rapids.ai/api/cuml/stable/
Apache License 2.0
4.27k stars 535 forks source link

[BUG]Unexpectedly High Forecast Values in Batch Prediction with cuML's Auto ARIMA #5838

Open Apolsus opened 7 months ago

Apolsus commented 7 months ago

Describe the bug I'm having some issues using cuML's Auto ARIMA model for large-scale time series forecasting. Specifically, when I tried to do a batch forecast on about 50,000 time series data, I got some unusually high values ​​in the forecast results. However, when I select the unusually sequence from these data and predict it alone, I can get normal prediction results. Steps/Code to reproduce bug Hard to discribe it here, the data is private and large. One of the sequence is: [224, 69, 115, 94, 59, 63, 60, 52, 87, 118, 132, 149, 139, 89, 97, 115, 98, 82, 55, 77, 96, 133, 112, 92, 170, 128, 94, 84, 63, 75, 56, 77, 85, 121, 126, 101, 197, 98, 89, 71, 72, 30, 47, 73, 69, 106, 110, 128]

batch prediction gives '7405687891.374923'

Expected behavior The results of individual predictions and batch predictions should be the same. Environment details (please complete the following information):

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
_sysroot_linux-64_curr_repodata_hack 3                   h69a702a_14    conda-forge
aiohttp                   3.9.3                    pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
alsa-lib                  1.2.11               hd590300_1    conda-forge
anyio                     4.3.0                    pypi_0    pypi
argon2-cffi               23.1.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
arrow                     1.3.0                    pypi_0    pypi
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
async-timeout             4.0.3                    pypi_0    pypi
attrs                     23.2.0                   pypi_0    pypi
beautifulsoup4            4.12.3                   pypi_0    pypi
binutils                  2.40                 hdd6e379_0    conda-forge
binutils_impl_linux-64    2.40                 hf600244_0    conda-forge
binutils_linux-64         2.40                 hdade7a5_3    conda-forge
bleach                    6.1.0                    pypi_0    pypi
bokeh                     3.3.4                    pypi_0    pypi
bzip2                     1.0.8                hd590300_5    conda-forge
c-compiler                1.7.0                hd590300_0    conda-forge
ca-certificates           2024.2.2             hbcca054_0    conda-forge
cachetools                5.3.3                    pypi_0    pypi
certifi                   2024.2.2                 pypi_0    pypi
cffi                      1.16.0                   pypi_0    pypi
charset-normalizer        3.3.2                    pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
click-plugins             1.1.1                    pypi_0    pypi
cligj                     0.7.2                    pypi_0    pypi
cloudpickle               3.0.0                    pypi_0    pypi
colorcet                  3.1.0                    pypi_0    pypi
comm                      0.2.2              pyhd8ed1ab_0    conda-forge
contourpy                 1.2.0                    pypi_0    pypi
cuda-cccl                 12.4.99              ha770c72_0    conda-forge
cuda-cccl-impl            2.0.1                ha770c72_1    conda-forge
cuda-cccl_linux-64        12.4.99              ha770c72_0    conda-forge
cuda-command-line-tools   12.4.0               ha770c72_0    conda-forge
cuda-compiler             12.4.0               hbad6d8a_0    conda-forge
cuda-crt-dev_linux-64     12.4.99              ha770c72_0    conda-forge
cuda-crt-tools            12.4.99              ha770c72_0    conda-forge
cuda-cudart               12.4.99              hd3aeb46_0    conda-forge
cuda-cudart-dev           12.4.99              hd3aeb46_0    conda-forge
cuda-cudart-dev_linux-64  12.4.99              h59595ed_0    conda-forge
cuda-cudart-static        12.4.99              hd3aeb46_0    conda-forge
cuda-cudart-static_linux-64 12.4.99              h59595ed_0    conda-forge
cuda-cudart_linux-64      12.4.99              h59595ed_0    conda-forge
cuda-cuobjdump            12.4.99              h59595ed_0    conda-forge
cuda-cupti                12.4.99              h59595ed_0    conda-forge
cuda-cupti-dev            12.4.99              h59595ed_0    conda-forge
cuda-cuxxfilt             12.4.99              h59595ed_0    conda-forge
cuda-documentation        12.2.53                       0    nvidia/label/cuda-12.2.0
cuda-driver-dev           12.4.99              hd3aeb46_0    conda-forge
cuda-driver-dev_linux-64  12.4.99              h59595ed_0    conda-forge
cuda-gdb                  12.4.99              hd47b8d6_0    conda-forge
cuda-libraries            12.4.0               ha770c72_0    conda-forge
cuda-libraries-dev        12.4.0               ha770c72_0    conda-forge
cuda-libraries-static     12.4.0               ha770c72_0    conda-forge
cuda-nsight               12.4.99              ha770c72_0    conda-forge
cuda-nvcc                 12.4.99              hcdd1206_0    conda-forge
cuda-nvcc-dev_linux-64    12.4.99              ha770c72_0    conda-forge
cuda-nvcc-impl            12.4.99              hd3aeb46_0    conda-forge
cuda-nvcc-tools           12.4.99              hd3aeb46_0    conda-forge
cuda-nvcc_linux-64        12.4.99              h8a487aa_0    conda-forge
cuda-nvdisasm             12.4.99              h59595ed_0    conda-forge
cuda-nvml-dev             12.4.99              h59595ed_0    conda-forge
cuda-nvprof               12.4.99              h59595ed_0    conda-forge
cuda-nvprune              12.4.99              h59595ed_0    conda-forge
cuda-nvrtc                12.4.99              hd3aeb46_0    conda-forge
cuda-nvrtc-dev            12.4.99              hd3aeb46_0    conda-forge
cuda-nvrtc-static         12.4.99              hd3aeb46_0    conda-forge
cuda-nvtx                 12.4.99              h59595ed_0    conda-forge
cuda-nvvm-dev_linux-64    12.4.99              ha770c72_0    conda-forge
cuda-nvvm-impl            12.4.99              h59595ed_0    conda-forge
cuda-nvvm-tools           12.4.99              h59595ed_0    conda-forge
cuda-nvvp                 12.4.99              h59595ed_0    conda-forge
cuda-opencl               12.4.99              h59595ed_0    conda-forge
cuda-opencl-dev           12.4.99              h59595ed_0    conda-forge
cuda-profiler-api         12.4.99              ha770c72_0    conda-forge
cuda-python               12.4.0                   pypi_0    pypi
cuda-sanitizer-api        12.4.99              hd3aeb46_0    conda-forge
cuda-toolkit              12.2.0                        0    nvidia/label/cuda-12.2.0
cuda-tools                12.4.0               ha770c72_0    conda-forge
cuda-version              12.4                 h3060b56_3    conda-forge
cuda-visual-tools         12.4.0               ha770c72_0    conda-forge
cudf-cu12                 24.4.0a580               pypi_0    pypi
cuml-cu12                 24.4.0a45                pypi_0    pypi
cuproj-cu12               24.4.0a28                pypi_0    pypi
cupy-cuda12x              13.0.0                   pypi_0    pypi
cuspatial-cu12            24.4.0a28                pypi_0    pypi
cuxfilter-cu12            24.4.0a15                pypi_0    pypi
cxx-compiler              1.7.0                h00ab1b0_0    conda-forge
dask                      2024.1.1                 pypi_0    pypi
dask-cuda                 24.4.0a12                pypi_0    pypi
dask-cudf-cu12            24.4.0a580               pypi_0    pypi
dask-expr                 0.4.0                    pypi_0    pypi
datashader                0.16.0                   pypi_0    pypi
dbus                      1.13.6               h5008d03_3    conda-forge
debugpy                   1.8.1            py39h3d6467e_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1                    pypi_0    pypi
distributed               2024.1.1                 pypi_0    pypi
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
expat                     2.6.2                h59595ed_0    conda-forge
fastjsonschema            2.19.1                   pypi_0    pypi
fastrlock                 0.8.2                    pypi_0    pypi
fiona                     1.9.6                    pypi_0    pypi
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_1    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fqdn                      1.5.1                    pypi_0    pypi
freetype                  2.12.1               h267a509_2    conda-forge
frozenlist                1.4.1                    pypi_0    pypi
fsspec                    2024.3.1                 pypi_0    pypi
gcc                       12.3.0               h95e488c_3    conda-forge
gcc_impl_linux-64         12.3.0               he2b93b0_5    conda-forge
gcc_linux-64              12.3.0               h6477408_3    conda-forge
gds-tools                 1.9.0.20             hd3aeb46_0    conda-forge
geopandas                 0.14.3                   pypi_0    pypi
gmp                       6.3.0                h59595ed_1    conda-forge
gxx                       12.3.0               h95e488c_3    conda-forge
gxx_impl_linux-64         12.3.0               he2b93b0_5    conda-forge
gxx_linux-64              12.3.0               h4a1b8e8_3    conda-forge
holoviews                 1.18.3                   pypi_0    pypi
icu                       73.2                 h59595ed_0    conda-forge
idna                      3.6                      pypi_0    pypi
importlib-metadata        7.0.2              pyha770c72_0    conda-forge
importlib_metadata        7.0.2                hd8ed1ab_0    conda-forge
ipykernel                 6.29.3             pyhd33586a_0    conda-forge
ipython                   8.18.1             pyh707e725_3    conda-forge
isoduration               20.11.0                  pypi_0    pypi
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.3                    pypi_0    pypi
joblib                    1.3.2                    pypi_0    pypi
jsonpointer               2.4                      pypi_0    pypi
jsonschema                4.21.1                   pypi_0    pypi
jsonschema-specifications 2023.12.1                pypi_0    pypi
jupyter-events            0.10.0                   pypi_0    pypi
jupyter-server            2.13.0                   pypi_0    pypi
jupyter-server-proxy      4.1.2                    pypi_0    pypi
jupyter-server-terminals  0.5.3                    pypi_0    pypi
jupyter_client            8.6.1              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2            py39hf3d152e_0    conda-forge
jupyterlab-pygments       0.3.0                    pypi_0    pypi
kernel-headers_linux-64   4.18.0               he073ed8_1    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
libcublas                 12.4.2.65            hd3aeb46_0    conda-forge
libcublas-dev             12.4.2.65            hd3aeb46_0    conda-forge
libcublas-static          12.4.2.65            hd3aeb46_0    conda-forge
libcufft                  11.2.0.44            hd3aeb46_0    conda-forge
libcufft-dev              11.2.0.44            hd3aeb46_0    conda-forge
libcufft-static           11.2.0.44            hd3aeb46_0    conda-forge
libcufile                 1.9.0.20             hd3aeb46_0    conda-forge
libcufile-dev             1.9.0.20             hd3aeb46_0    conda-forge
libcufile-static          1.9.0.20             hd3aeb46_0    conda-forge
libcurand                 10.3.5.119           hd3aeb46_0    conda-forge
libcurand-dev             10.3.5.119           hd3aeb46_0    conda-forge
libcurand-static          10.3.5.119           hd3aeb46_0    conda-forge
libcusolver               11.6.0.99            hd3aeb46_0    conda-forge
libcusolver-dev           11.6.0.99            hd3aeb46_0    conda-forge
libcusolver-static        11.6.0.99            hd3aeb46_0    conda-forge
libcusparse               12.3.0.142           hd3aeb46_0    conda-forge
libcusparse-dev           12.3.0.142           hd3aeb46_0    conda-forge
libcusparse-static        12.3.0.142           hd3aeb46_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libexpat                  2.6.2                h59595ed_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-devel_linux-64     12.3.0             h8bca6fd_105    conda-forge
libgcc-ng                 13.2.0               h807b86a_5    conda-forge
libglib                   2.80.0               hf2295e7_0    conda-forge
libgomp                   13.2.0               h807b86a_5    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libnpp                    12.2.5.2             hd3aeb46_0    conda-forge
libnpp-dev                12.2.5.2             hd3aeb46_0    conda-forge
libnpp-static             12.2.5.2             hd3aeb46_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libnuma                   2.0.18               hd590300_0    conda-forge
libnvfatbin               12.4.99              hd3aeb46_0    conda-forge
libnvfatbin-dev           12.4.99              hd3aeb46_0    conda-forge
libnvfatbin-static        12.4.99              hd3aeb46_0    conda-forge
libnvjitlink              12.4.99              hd3aeb46_0    conda-forge
libnvjitlink-dev          12.4.99              hd3aeb46_0    conda-forge
libnvjitlink-static       12.4.99              hd3aeb46_0    conda-forge
libnvjpeg                 12.3.1.89            h59595ed_0    conda-forge
libnvjpeg-dev             12.3.1.89            ha770c72_0    conda-forge
libnvjpeg-static          12.3.1.89            ha770c72_0    conda-forge
libpng                    1.6.43               h2797004_0    conda-forge
libsanitizer              12.3.0               h0f45ef3_5    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.45.2               h2797004_0    conda-forge
libstdcxx-devel_linux-64  12.3.0             h8bca6fd_105    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_5    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxkbcommon              1.6.0                hd429924_1    conda-forge
libxkbfile                1.1.0                h166bdaf_1    conda-forge
libxml2                   2.12.6               h232c23b_0    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
linkify-it-py             2.0.3                    pypi_0    pypi
llvmlite                  0.42.0                   pypi_0    pypi
locket                    1.0.0                    pypi_0    pypi
markdown                  3.6                      pypi_0    pypi
markdown-it-py            3.0.0                    pypi_0    pypi
markupsafe                2.1.5                    pypi_0    pypi
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mdit-py-plugins           0.4.0                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
mistune                   3.0.2                    pypi_0    pypi
msgpack                   1.0.8                    pypi_0    pypi
multidict                 6.0.5                    pypi_0    pypi
multipledispatch          1.0.0                    pypi_0    pypi
nbclient                  0.10.0                   pypi_0    pypi
nbconvert                 7.16.2                   pypi_0    pypi
nbformat                  5.10.3                   pypi_0    pypi
ncurses                   6.4.20240210         h59595ed_0    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
nsight-compute            2024.1.0.13          hf972014_1    conda-forge
nspr                      4.35                 h27087fc_0    conda-forge
nss                       3.98                 h1d7d5a4_0    conda-forge
numba                     0.59.1                   pypi_0    pypi
numpy                     1.26.4                   pypi_0    pypi
nvtx                      0.2.10                   pypi_0    pypi
ocl-icd                   2.3.2                hd590300_0    conda-forge
openssl                   3.2.1                hd590300_1    conda-forge
overrides                 7.7.0                    pypi_0    pypi
packaging                 24.0               pyhd8ed1ab_0    conda-forge
pandas                    2.2.1                    pypi_0    pypi
pandocfilters             1.5.1                    pypi_0    pypi
panel                     1.3.8                    pypi_0    pypi
param                     2.0.2                    pypi_0    pypi
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
partd                     1.4.1                    pypi_0    pypi
patsy                     0.5.6                    pypi_0    pypi
pcre2                     10.43                hcad00b1_0    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.2.0                   pypi_0    pypi
pip                       24.0               pyhd8ed1ab_0    conda-forge
platformdirs              4.2.0              pyhd8ed1ab_0    conda-forge
prometheus-client         0.20.0                   pypi_0    pypi
prompt-toolkit            3.0.42             pyha770c72_0    conda-forge
protobuf                  4.25.3                   pypi_0    pypi
psutil                    5.9.8            py39hd1e30aa_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pyarrow                   14.0.2                   pypi_0    pypi
pycparser                 2.21                     pypi_0    pypi
pyct                      0.5.0                    pypi_0    pypi
pygments                  2.17.2             pyhd8ed1ab_0    conda-forge
pylibraft-cu12            24.4.0a88                pypi_0    pypi
pynvjitlink-cu12          0.1.14                   pypi_0    pypi
pynvml                    11.4.1                   pypi_0    pypi
pyproj                    3.6.1                    pypi_0    pypi
python                    3.9.18          h0755675_1_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7                    pypi_0    pypi
python_abi                3.9                      4_cp39    conda-forge
pytz                      2024.1                   pypi_0    pypi
pyviz-comms               3.0.1                    pypi_0    pypi
pyyaml                    6.0.1                    pypi_0    pypi
pyzmq                     25.1.2           py39h8c080ef_0    conda-forge
raft-dask-cu12            24.4.0a88                pypi_0    pypi
rapids-dask-dependency    24.4.0a7                 pypi_0    pypi
readline                  8.2                  h8228510_1    conda-forge
referencing               0.34.0                   pypi_0    pypi
requests                  2.31.0                   pypi_0    pypi
rfc3339-validator         0.1.4                    pypi_0    pypi
rfc3986-validator         0.1.1                    pypi_0    pypi
rich                      13.7.1                   pypi_0    pypi
rmm-cu12                  24.4.0a38                pypi_0    pypi
rpds-py                   0.18.0                   pypi_0    pypi
scipy                     1.12.0                   pypi_0    pypi
send2trash                1.8.2                    pypi_0    pypi
setuptools                69.2.0             pyhd8ed1ab_0    conda-forge
shapely                   2.0.3                    pypi_0    pypi
simpervisor               1.0.0                    pypi_0    pypi
six                       1.16.0             pyh6c4a22f_0    conda-forge
sniffio                   1.3.1                    pypi_0    pypi
sortedcontainers          2.4.0                    pypi_0    pypi
soupsieve                 2.5                      pypi_0    pypi
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
statsmodels               0.14.1                   pypi_0    pypi
sysroot_linux-64          2.28                 he073ed8_1    conda-forge
tblib                     3.0.0                    pypi_0    pypi
terminado                 0.18.1                   pypi_0    pypi
tinycss2                  1.2.1                    pypi_0    pypi
tk                        8.6.13          noxft_h4845f30_101    conda-forge
toolz                     0.12.1                   pypi_0    pypi
tornado                   6.4              py39hd1e30aa_0    conda-forge
tqdm                      4.66.2                   pypi_0    pypi
traitlets                 5.14.2             pyhd8ed1ab_0    conda-forge
treelite                  4.0.0                    pypi_0    pypi
types-python-dateutil     2.9.0.20240316           pypi_0    pypi
typing_extensions         4.10.0             pyha770c72_0    conda-forge
tzdata                    2024.1                   pypi_0    pypi
uc-micro-py               1.0.3                    pypi_0    pypi
ucx-py-cu12               0.37.0a15                pypi_0    pypi
uri-template              1.3.0                    pypi_0    pypi
urllib3                   2.2.1                    pypi_0    pypi
wayland                   1.22.0               h8c25dac_1    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
webcolors                 1.13                     pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.7.0                    pypi_0    pypi
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
xarray                    2024.2.0                 pypi_0    pypi
xcb-util                  0.4.0                hd590300_1    conda-forge
xcb-util-cursor           0.1.4                hd590300_1    conda-forge
xcb-util-image            0.4.0                h8ee46fc_1    conda-forge
xcb-util-keysyms          0.4.0                h8ee46fc_1    conda-forge
xcb-util-renderutil       0.3.9                hd590300_1    conda-forge
xcb-util-wm               0.4.1                h8ee46fc_1    conda-forge
xkeyboard-config          2.41                 hd590300_0    conda-forge
xorg-compositeproto       0.4.2             h7f98852_1001    conda-forge
xorg-damageproto          1.2.1             h7f98852_1002    conda-forge
xorg-fixesproto           5.0               h7f98852_1002    conda-forge
xorg-inputproto           2.3.2             h7f98852_1002    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.7                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxcomposite        0.4.6                h0b41bf4_1    conda-forge
xorg-libxdamage           1.1.5                h7f98852_1    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxfixes            5.0.3             h7f98852_1004    conda-forge
xorg-libxi                1.7.10               h7f98852_0    conda-forge
xorg-libxrandr            1.5.2                h7f98852_1    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-libxtst              1.2.3             h7f98852_1002    conda-forge
xorg-randrproto           1.5.0             h7f98852_1001    conda-forge
xorg-recordproto          1.14.2            h7f98852_1002    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-util-macros          1.19.3               h7f98852_0    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xyzservices               2023.10.1                pypi_0    pypi
xz                        5.2.6                h166bdaf_0    conda-forge
yarl                      1.9.4                    pypi_0    pypi
zeromq                    4.3.5                h59595ed_1    conda-forge
zict                      3.0.0                    pypi_0    pypi
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
Apolsus commented 7 months ago

After several attempts, I determined that the problem was that there were some problems with auto-arima's search. Even for the same sequence, different best model search results may appear under different batch combinations. Here is a minimal example.

ts = [
      [128, 88, 228, 357, 376, 448, 558, 521, 395, 211, 294, 253, 314, 272, 314, 295, 432, 565, 583, 473, 343, 195, 200, 255, 371, 189, 269, 272, 352, 374, 433, 296, 182, 187, 136, 82, 113, 105, 217, 203, 416, 504, 488, 519, 424, 263, 263, 220, 237, 160],
      [214, 1351, 965, 848, 362, 273, 176, 187, 163, 201, 207, 173, 239, 137, 233, 231, 248, 160, 200, 222, 160,219, 173, 170, 225, 212, 216, 279, 197, 174, 163, 184, 149, 169, 178, 181, 212, 177, 172, 224, 234, 225, 204, 165, 178, 163, 140, 138, 125, 175],
]
ys = cupy.array(ts)
ys = ys.T
model = AutoARIMA(ys)
model.search(s = 0, d=(0, 1, 2), p=range(7), q=range(4), method="auto")
model.fit()
model.forecast(2)
ts = [
      [214, 1351, 965, 848, 362, 273, 176, 187, 163, 201, 207, 173, 239, 137, 233, 231, 248, 160, 200, 222, 160,219, 173, 170, 225, 212, 216, 279, 197, 174, 163, 184, 149, 169, 178, 181, 212, 177, 172, 224, 234, 225, 204, 165, 178, 163, 140, 138, 125, 175],
]
ys = cupy.array(ts)
ys = ys.T
model = AutoARIMA(ys)
model.search(s = 0, d=(0, 1, 2), p=range(7), q=range(4), method="auto")
model.fit()
model.forecast(2)
Apolsus commented 7 months ago

@Nyrio I noticed that you are the main contributor of this code, could you provide some help and mark this as a feature or I can also try to fix this bug.

Apolsus commented 7 months ago

This problem is further confirmed to be that when ARIMA(method='ml'), the fitting results of batch input and individual input are different, but method='css' does not have this problem.

dantegd commented 7 months ago

@Apolsus thanks for the issue and reproducer! Would using the css method suffice for you for now? We will look into the bug in ml, but not sure of an ETA for a fix

Apolsus commented 7 months ago

no, css method will also cause this in a much larger batchsize (10000). I checked the source code. In theory, the parameter optimization process is performed independently for each sequence, but some sequences do not converge in batch prediction, but converge in individual prediction. I think one possible place is the parameter initialization process, I haven't had time to check yet.

dantegd commented 7 months ago

After looking into it (and with help from @Nyrio ) this seems to stem from numerical stability issues particularly around different code paths for different batch sizes. It might take some time to create workarounds or fixes in general, but we will try to look into it as soon as we can.