Cannot Install on Apple M1

Describe the bug I cannot install tslearn on Apple Silicon M1. It's trying to gather numpy >= 1.19.0 which does not yet have automatic support for M1. But as of version 1.20 they include automatic support for M1 I believe.

To Reproduce Poetry install with tslearn as a dependency

Expected behavior Successful install

Environment (please complete the following information):

OS: macOS 12.4 (Monterey)
Chip: Apple M1
Python 3.10.5
poetry 1.1.11
pip 22.2.2
tslearn version 0.5.2

Additional context Add any other context about the problem here.

$ poetry install
Installing dependencies from lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing tslearn (0.5.2): Failed

  EnvCommandError

  Command ['~/.venv/bin/pip', 'install', '--no-deps', 'file://~/Library/Caches/pypoetry/artifacts/51/d8/00/a61346977edfe32b97cc26c63e6cd8474c1604a62460dc3db7cb01d92d/tslearn-0.5.2.tar.gz'] errored with the following return code 1, and output:
  Processing ~/Library/Caches/pypoetry/artifacts/51/d8/00/a61346977edfe32b97cc26c63e6cd8474c1604a62460dc3db7cb01d92d/tslearn-0.5.2.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'error'
    error: subprocess-exited-with-error

    × pip subprocess to install build dependencies did not run successfully.
    │ exit code: 1
    ╰─> [51 lines of output]
        Collecting setuptools
          Using cached setuptools-65.0.0-py3-none-any.whl (1.2 MB)
        Collecting wheel
          Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
        Collecting numpy<=1.19
          Using cached numpy-1.19.0.zip (7.3 MB)
          Installing build dependencies: started
          Installing build dependencies: finished with status 'done'
          Getting requirements to build wheel: started
          Getting requirements to build wheel: finished with status 'done'
          Preparing metadata (pyproject.toml): started
          Preparing metadata (pyproject.toml): finished with status 'error'
          error: subprocess-exited-with-error

          × Preparing metadata (pyproject.toml) did not run successfully.
          │ exit code: 1
          ╰─> [24 lines of output]
              Running from numpy source directory.
              <string>:460: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
              Traceback (most recent call last):
                File "~/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
                  main()
                File "~/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
                  json_out['return_val'] = hook(**hook_input['kwargs'])
                File "~/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
                  return hook(metadata_directory, config_settings)
                File "/private/var/folders/m5/v1lrl3292y3191v_0hsxs8380000gn/T/pip-build-env-n1x8il70/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 369, in prepare_metadata_for_build_wheel
                  self.run_setup()
                File "/private/var/folders/m5/v1lrl3292y3191v_0hsxs8380000gn/T/pip-build-env-n1x8il70/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 474, in run_setup
                  super(_BuildMetaLegacyBackend,
                File "/private/var/folders/m5/v1lrl3292y3191v_0hsxs8380000gn/T/pip-build-env-n1x8il70/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 334, in run_setup
                  exec(code, locals())
                File "<string>", line 489, in <module>
                File "<string>", line 465, in setup_package
                File "/private/var/folders/m5/v1lrl3292y3191v_0hsxs8380000gn/T/pip-install-948u71m0/numpy_d47a0d0859494031959537ee72b429fa/numpy/distutils/core.py", line 24, in <module>
                  from numpy.distutils.command import config, config_compiler, \
                File "/private/var/folders/m5/v1lrl3292y3191v_0hsxs8380000gn/T/pip-install-948u71m0/numpy_d47a0d0859494031959537ee72b429fa/numpy/distutils/command/config.py", line 19, in <module>
                  from numpy.distutils.mingw32ccompiler import generate_manifest
                File "/private/var/folders/m5/v1lrl3292y3191v_0hsxs8380000gn/T/pip-install-948u71m0/numpy_d47a0d0859494031959537ee72b429fa/numpy/distutils/mingw32ccompiler.py", line 28, in <module>
                  from distutils.msvccompiler import get_build_version as get_build_msvc_version
              ModuleNotFoundError: No module named 'distutils.msvccompiler'
              [end of output]

          note: This error originates from a subprocess, and is likely not a problem with pip.
        error: metadata-generation-failed

        × Encountered error while generating package metadata.
        ╰─> See above for output.

        note: This is an issue with the package mentioned above, not pip.
        hint: See above for details.
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.

I think if we bump the numpy dependency to >=1.20.0 It will automatically pickup a numpy wheel with arm64 support.

Edit: my original comment below installs tslearn without error, but then there is an error when doing an import, like from tslearn.clustering import TimeSeriesKMeans, similar to what is mentioned in #419. The error is

Traceback (most recent call last):
  File "__init__.pxd", line 942, in numpy.import_array
RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/me/opt/miniconda3/envs/tsl/lib/python3.10/site-packages/tslearn/clustering/__init__.py", line 8, in <module>
    from .kshape import KShape
  File "/Users/me/opt/miniconda3/envs/tsl/lib/python3.10/site-packages/tslearn/clustering/kshape.py", line 11, in <module>
    from tslearn.metrics import cdist_normalized_cc, y_shifted_sbd_vec
  File "/Users/me/opt/miniconda3/envs/tsl/lib/python3.10/site-packages/tslearn/metrics/__init__.py", line 19, in <module>
    from .softdtw_variants import (cdist_soft_dtw, cdist_gak,
  File "/Users/me/opt/miniconda3/envs/tsl/lib/python3.10/site-packages/tslearn/metrics/softdtw_variants.py", line 12, in <module>
    from .soft_dtw_fast import _soft_dtw, _soft_dtw_grad, \
  File "tslearn/metrics/soft_dtw_fast.pyx", line 11, in init tslearn.metrics.soft_dtw_fast
  File "__init__.pxd", line 944, in numpy.import_array
ImportError: numpy.core.multiarray failed to import

The same thing happens when installing with

python -m pip install https://github.com/tslearn-team/tslearn/archive/main.zip

I've tried Python versions 3.8.13, 3.9.12, and 3.10.4. Upgrading pip and uninstalling and reinstalling numpy version 1.22.4 did not resolve the problem.

Original comment:

I have the same problem when trying to install v0.5.2 on an M1 Mac. A workaround is to install from the repository:

pip install git+https://github.com/tslearn-team/tslearn

This commit removed the numpy<=1.19 requirement, but it hasn't been released in an updated version, so I don't think it's on PyPI yet.

Installing from the GitHub repository with the below command also leads to importing errors on my older, Intel Mac.

python -m pip install -v https://github.com/tslearn-team/tslearn/archive/main.zip

I believe the problem is that numpy-1.23.2 is installed as the build dependency, but numpy-1.22.4 is installed as the runtime dependency. By this, I mean that numpy-1.23.2 is used in setup.py , such as in these lines

include_dirs=[numpy.get_include()], 
ext_modules=cythonize("tslearn/metrics/*.pyx",
                  include_path=[numpy.get_include()]),

but then numpy-1.22.4 is installed in the user’s environment, so that when the user is running Python and importing tslearn, they are using numpy-1.22.4. I believe this is the source of the error noted above: “RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf”

pyproject.toml only specifies “numpy”, so the latest version is used to build tslearn. For the runtime dependencies, numba doesn’t support numpy 1.23 yet, so 1.22.4 is installed.

I was able to get things working by updating pyproject.toml to

[build-system]
requires = ["setuptools", "wheel", "numpy<1.23", "Cython"]

However, I’m not sure this is the best fix since it would probably need to be updated again once numba supports numpy 1.23.

Another fix, which is suggested in this pip issue is to use “oldest-supported-numpy” in pyproject.toml:

[build-system]
requires = ["setuptools", "wheel", "oldest-supported-numpy", "Cython"]

This works on both my Intel and M1 MacBooks.

@rtavenar Would you be receptive to a PR that makes the above changes to pyproject.toml to address this issue?

Doesn't seem like a complicated fix. Could someone please explain the holdup and if I could help in any way to speed things up? @DanielKerrigan @wangjoshuah

tslearn-team / tslearn

Cannot Install on Apple M1 #418