search

alkaline-ml / pmdarima

A statistical library designed to fill the void in Python's time series analysis capabilities, including the equivalent of R's auto.arima function.
https://www.alkaline-ml.com/pmdarima
MIT License
1.6k stars 234 forks source link

`pip install git+https://github.com/alkaline-ml/pmdarima.git` throws `ModuleNotFoundError: No module named 'distutils.msvccompiler'` #582

Open harupy opened 2 months ago

harupy commented 2 months ago

This is not a bug. In https://github.com/mlflow/mlflow, we use pip install git+https://github.com/alkaline-ml/pmdarima.git to install the development version of pmdarima and run tests against it. This command started failing with the following error recently:

% docker run --rm python:3.8 pip install git+https://github.com/harupy/pmdarima.git@pin-setuptools
Collecting git+https://github.com/harupy/pmdarima.git@pin-setuptools
  Cloning https://github.com/harupy/pmdarima.git (to revision pin-setuptools) to /tmp/pip-req-build-m7yplexv
  Running command git clone --filter=blob:none --quiet https://github.com/harupy/pmdarima.git /tmp/pip-req-build-m7yplexv
  Running command git checkout -b pin-setuptools --track origin/pin-setuptools
  Switched to a new branch 'pin-setuptools'
  branch 'pin-setuptools' set up to track 'origin/pin-setuptools'.
  Resolved https://github.com/harupy/pmdarima.git to commit 0e207106ce8562a1db57a0ff19b794255d8ae062
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: setuptools!=50.0.0,>=38.6.0 in /usr/local/lib/python3.8/site-packages (from pmdarima==0.0.0) (57.5.0)
Collecting packaging>=17.1
  Using cached packaging-24.1-py3-none-any.whl (53 kB)
Collecting pandas>=0.19
  Using cached pandas-2.0.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.7 MB)
Collecting Cython!=0.29.18,!=0.29.31,>=0.29
  Using cached Cython-3.0.11-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB)
Collecting scikit-learn>=0.22
  Downloading scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.5 MB/s eta 0:00:00
Collecting scipy>=1.3.2
  Downloading scipy-1.10.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (31.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.0/31.0 MB 10.4 MB/s eta 0:00:00
Collecting statsmodels>=0.13.2
  Downloading statsmodels-0.14.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.5 MB/s eta 0:00:00
Collecting numpy>=1.21.2
  Downloading numpy-1.24.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 10.3 MB/s eta 0:00:00
Collecting urllib3
  Downloading urllib3-2.2.3-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.3/126.3 kB 10.3 MB/s eta 0:00:00
Collecting joblib>=0.11
  Downloading joblib-1.4.2-py3-none-any.whl (301 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 301.8/301.8 kB 9.8 MB/s eta 0:00:00
Collecting python-dateutil>=2.8.2
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Collecting pytz>=2020.1
  Using cached pytz-2024.2-py2.py3-none-any.whl (508 kB)
Collecting tzdata>=2022.1
  Using cached tzdata-2024.1-py2.py3-none-any.whl (345 kB)
Collecting threadpoolctl>=2.0.0
  Downloading threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
Collecting patsy>=0.5.4
  Using cached patsy-0.5.6-py2.py3-none-any.whl (233 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: pmdarima
  Building wheel for pmdarima (pyproject.toml): started
  Building wheel for pmdarima (pyproject.toml): finished with status 'done'
  Created wheel for pmdarima: filename=pmdarima-0.0.0-cp38-cp38-linux_aarch64.whl size=2104950 sha256=4d028a38259c984bc32b523d18e8910c5411d8d4a38d63216d083fec5003b6b5
  Stored in directory: /tmp/pip-ephem-wheel-cache-54mjx6wz/wheels/b9/40/4a/88c839cd3c48e189fdc633b304d2bb581af11d55d1f2f7ee37
Successfully built pmdarima
Installing collected packages: pytz, urllib3, tzdata, threadpoolctl, six, packaging, numpy, joblib, Cython, scipy, python-dateutil, patsy, scikit-learn, pandas, statsmodels, pmdarima
Successfully installed Cython-3.0.11 joblib-1.4.2 numpy-1.24.4 packaging-24.1 pandas-2.0.3 patsy-0.5.6 pmdarima-0.0.0 python-dateutil-2.9.0.post0 pytz-2024.2 scikit-learn-1.3.2 scipy-1.10.1 six-1.16.0 statsmodels-0.14.1 threadpoolctl-3.5.0 tzdata-2024.1 urllib3-2.2.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: pip install --upgrade pip

(mlflow) @ mlflow [pmdarima-python-3.9] -- INSERT --
% docker run --rm python:3.8 pip install git+https://github.com/alkaline-ml/pmdarima.git
Collecting git+https://github.com/alkaline-ml/pmdarima.git
  Cloning https://github.com/alkaline-ml/pmdarima.git to /tmp/pip-req-build-0gm1driz
  Running command git clone --filter=blob:none --quiet https://github.com/alkaline-ml/pmdarima.git /tmp/pip-req-build-0gm1driz
  Resolved https://github.com/alkaline-ml/pmdarima.git to commit c33a01768dfaf65cadfa666420e75548b8587011
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting joblib>=0.11
  Downloading joblib-1.4.2-py3-none-any.whl (301 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 301.8/301.8 kB 6.7 MB/s eta 0:00:00
Collecting Cython!=0.29.18,!=0.29.31,>=0.29
  Using cached Cython-3.0.11-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB)
Collecting numpy>=1.21.2
  Downloading numpy-1.24.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 10.4 MB/s eta 0:00:00
Collecting scipy>=1.3.2
  Downloading scipy-1.10.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (31.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.0/31.0 MB 10.3 MB/s eta 0:00:00
Requirement already satisfied: setuptools!=50.0.0,>=38.6.0 in /usr/local/lib/python3.8/site-packages (from pmdarima==0.0.0) (57.5.0)
Collecting pandas>=0.19
  Using cached pandas-2.0.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.7 MB)
Collecting urllib3
  Downloading urllib3-2.2.3-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.3/126.3 kB 10.7 MB/s eta 0:00:00
Collecting statsmodels>=0.13.2
  Downloading statsmodels-0.14.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.4 MB/s eta 0:00:00
Collecting packaging>=17.1
  Using cached packaging-24.1-py3-none-any.whl (53 kB)
Collecting scikit-learn>=0.22
  Downloading scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.4 MB/s eta 0:00:00
Collecting pytz>=2020.1
  Using cached pytz-2024.2-py2.py3-none-any.whl (508 kB)
Collecting python-dateutil>=2.8.2
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Collecting tzdata>=2022.1
  Using cached tzdata-2024.1-py2.py3-none-any.whl (345 kB)
Collecting threadpoolctl>=2.0.0
  Downloading threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
Collecting patsy>=0.5.4
  Using cached patsy-0.5.6-py2.py3-none-any.whl (233 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: pmdarima
  Building wheel for pmdarima (pyproject.toml): started
  Building wheel for pmdarima (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error

  × Building wheel for pmdarima (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [27 lines of output]
      Partial import of pmdarima during the build process.
      Requirements: ['joblib>=0.11', 'Cython>=0.29,!=0.29.18,!=0.29.31', 'numpy>=1.21.2', 'pandas>=0.19', 'scikit-learn>=0.22', 'scipy>=1.3.2', 'statsmodels>=0.13.2', 'urllib3', 'setuptools>=38.6.0,!=50.0.0', 'packaging>=17.1  # Bundled with setuptools, but want to be explicit']
      Adding extra setuptools args
      Traceback (most recent call last):
        File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 421, in build_wheel
          return self._build_with_temp_dir(
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 403, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 503, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 318, in run_setup
          exec(code, locals())
        File "<string>", line 371, in <module>
        File "<string>", line 360, in do_setup
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/numpy/distutils/core.py", line 24, in <module>
          from numpy.distutils.command import config, config_compiler, \
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/numpy/distutils/command/config.py", line 19, in <module>
          from numpy.distutils.mingw32ccompiler import generate_manifest
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/numpy/distutils/mingw32ccompiler.py", line 29, in <module>
          from distutils.msvccompiler import get_build_version as get_build_msvc_version
      ModuleNotFoundError: No module named 'distutils.msvccompiler'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pmdarima
Failed to build pmdarima
ERROR: Could not build wheels for pmdarima, which is required to install pyproject.toml-based projects

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
harupy commented 2 months ago

I'm investigating a fix on https://github.com/harupy/pmdarima/tree/pin-setuptools.

https://github.com/harupy/pmdarima/commit/0e207106ce8562a1db57a0ff19b794255d8ae062 seems to work.

% docker run --rm python:3.8 pip install git+https://github.com/harupy/pmdarima.git@0e207106ce8562a1db57a0ff19b794255d8ae062
Collecting git+https://github.com/harupy/pmdarima.git@0e207106ce8562a1db57a0ff19b794255d8ae062
  Cloning https://github.com/harupy/pmdarima.git (to revision 0e207106ce8562a1db57a0ff19b794255d8ae062) to /tmp/pip-req-build-sr6q8f4e
  Running command git clone --filter=blob:none --quiet https://github.com/harupy/pmdarima.git /tmp/pip-req-build-sr6q8f4e
  Running command git rev-parse -q --verify 'sha^0e207106ce8562a1db57a0ff19b794255d8ae062'
  Running command git fetch -q https://github.com/harupy/pmdarima.git 0e207106ce8562a1db57a0ff19b794255d8ae062
  Running command git checkout -q 0e207106ce8562a1db57a0ff19b794255d8ae062
  Resolved https://github.com/harupy/pmdarima.git to commit 0e207106ce8562a1db57a0ff19b794255d8ae062
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting Cython!=0.29.18,!=0.29.31,>=0.29
  Using cached Cython-3.0.11-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB)
Collecting packaging>=17.1
  Using cached packaging-24.1-py3-none-any.whl (53 kB)
Collecting urllib3
  Downloading urllib3-2.2.3-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.3/126.3 kB 4.7 MB/s eta 0:00:00
Collecting numpy>=1.21.2
  Downloading numpy-1.24.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 10.3 MB/s eta 0:00:00
Collecting scikit-learn>=0.22
  Downloading scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.4 MB/s eta 0:00:00
Collecting joblib>=0.11
  Downloading joblib-1.4.2-py3-none-any.whl (301 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 301.8/301.8 kB 10.1 MB/s eta 0:00:00
Requirement already satisfied: setuptools!=50.0.0,>=38.6.0 in /usr/local/lib/python3.8/site-packages (from pmdarima==0.0.0) (57.5.0)
Collecting statsmodels>=0.13.2
  Downloading statsmodels-0.14.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.4 MB/s eta 0:00:00
Collecting pandas>=0.19
  Using cached pandas-2.0.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.7 MB)
Collecting scipy>=1.3.2
  Downloading scipy-1.10.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (31.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.0/31.0 MB 10.2 MB/s eta 0:00:00
Collecting tzdata>=2022.1
  Using cached tzdata-2024.1-py2.py3-none-any.whl (345 kB)
Collecting pytz>=2020.1
  Using cached pytz-2024.2-py2.py3-none-any.whl (508 kB)
Collecting python-dateutil>=2.8.2
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Collecting threadpoolctl>=2.0.0
  Downloading threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
Collecting patsy>=0.5.4
  Using cached patsy-0.5.6-py2.py3-none-any.whl (233 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: pmdarima
  Building wheel for pmdarima (pyproject.toml): started
  Building wheel for pmdarima (pyproject.toml): finished with status 'done'
  Created wheel for pmdarima: filename=pmdarima-0.0.0-cp38-cp38-linux_aarch64.whl size=2104962 sha256=130266f6b132ec5a8bc2f66a84f33af41a6a5868e25cf5e7d5bcd7ae2fbc0ef3
  Stored in directory: /root/.cache/pip/wheels/d4/b5/f7/3057ff446df49ded9a2d2898468572a9d244d3be0d27538486
Successfully built pmdarima
Installing collected packages: pytz, urllib3, tzdata, threadpoolctl, six, packaging, numpy, joblib, Cython, scipy, python-dateutil, patsy, scikit-learn, pandas, statsmodels, pmdarima
Successfully installed Cython-3.0.11 joblib-1.4.2 numpy-1.24.4 packaging-24.1 pandas-2.0.3 patsy-0.5.6 pmdarima-0.0.0 python-dateutil-2.9.0.post0 pytz-2024.2 scikit-learn-1.3.2 scipy-1.10.1 six-1.16.0 statsmodels-0.14.1 threadpoolctl-3.5.0 tzdata-2024.1 urllib3-2.2.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
harupy commented 2 months ago

Looks like this is an issue in numpy.

harupy commented 2 months ago

https://github.com/alkaline-ml/pmdarima/blob/c33a01768dfaf65cadfa666420e75548b8587011/pmdarima/_build_utils/__init__.py#L7 looks like the culprit.

harupy commented 2 months ago

Can we pin setuptools?

harupy commented 1 month ago

Any updates on this issue?