search

modin-project / modin

Modin: Scale your Pandas workflows by changing a single line of code
http://modin.readthedocs.io
Apache License 2.0
9.59k stars 647 forks source link

Installation of modin using pip fails #7282

Closed helloplayer1 closed 1 month ago

helloplayer1 commented 1 month ago

I wanted to install modin with the ray engine using pip: pip install "modin[ray]", but this fails with the following error:

Collecting pandas==1.3.3 (from modin[ray])
  Using cached pandas-1.3.3.tar.gz (4.7 MB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 2
  ╰─> [114 lines of output]
      Ignoring numpy: markers 'python_version == "3.7" and (platform_machine != "arm64" or platform_system != "Darwin") and platform_machine != "aarch64"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.8" and (platform_machine != "arm64" or platform_system != "Darwin") and platform_machine != "aarch64"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.7" and platform_machine == "aarch64"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.8" and platform_machine == "aarch64"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.8" and platform_machine == "arm64" and platform_system == "Darwin"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.9" and platform_machine == "arm64" and platform_system == "Darwin"' don't match your environment
      Collecting setuptools>=51.0.0
        Using cached setuptools-69.5.1-py3-none-any.whl.metadata (6.2 kB)
      Collecting wheel
        Using cached wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB)
      Collecting Cython<3,>=0.29.21
        Using cached Cython-0.29.37-py2.py3-none-any.whl.metadata (3.1 kB)
      Collecting numpy==1.19.3
        Using cached numpy-1.19.3.zip (7.3 MB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
      ERROR: Exception:
      Traceback (most recent call last):
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\cli\base_command.py", line 180, in exc_logging_wrapper
          status = run_func(*args)
                   ^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\cli\req_command.py", line 245, in wrapper
          return func(self, options, args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\commands\install.py", line 377, in run
          requirement_set = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 95, in resolve
          result = self._result = resolver.resolve(
                                  ^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 546, in resolve
          state = resolution.resolve(requirements, max_rounds=max_rounds)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 397, in resolve
          self._add_to_criteria(self.state.criteria, r, parent=None)
        File "D:\BAAI\.venv\Lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 173, in _add_to_criteria
          if not criterion.candidates:
                 ^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_vendor\resolvelib\structs.py", line 156, in __bool__
          return bool(self._sequence)
                 ^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 155, in __bool__
          return any(self)
                 ^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 143, in <genexpr>
          return (c for c in iterator if id(c) not in self._incompatible_ids)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 47, in _iter_built
          candidate = func()
                      ^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 182, in _make_candidate_from_link
          base: Optional[BaseCandidate] = self._make_base_candidate_from_link(
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 228, in _make_base_candidate_from_link
          self._link_candidate_cache[link] = LinkCandidate(
                                             ^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 290, in __init__
          super().__init__(
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 156, in __init__
          self.dist = self._prepare()
                      ^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 222, in _prepare
          dist = self._prepare_distribution()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 301, in _prepare_distribution
          return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\operations\prepare.py", line 525, in prepare_linked_requirement
          return self._prepare_linked_requirement(req, parallel_builds)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\operations\prepare.py", line 640, in _prepare_linked_requirement
          dist = _get_prepared_distribution(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\operations\prepare.py", line 71, in _get_prepared_distribution
          abstract_dist.prepare_distribution_metadata(
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\distributions\sdist.py", line 54, in prepare_distribution_metadata
          self._install_build_reqs(finder)
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\distributions\sdist.py", line 124, in _install_build_reqs
          build_reqs = self._get_build_requires_wheel()
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\distributions\sdist.py", line 101, in _get_build_requires_wheel
          return backend.get_requires_for_build_wheel()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_internal\utils\misc.py", line 745, in get_requires_for_build_wheel
          return super().get_requires_for_build_wheel(config_settings=cs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_impl.py", line 166, in get_requires_for_build_wheel
          return self._call_hook('get_requires_for_build_wheel', {
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\BAAI\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_impl.py", line 321, in _call_hook
          raise BackendUnavailable(data.get('traceback', ''))
      pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
        File "D:\BAAI\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 77, in _build_backend
          obj = import_module(mod_path)
                ^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Program Files\Python312\Lib\importlib\__init__.py", line 90, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
        File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
        File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
        File "<frozen importlib._bootstrap_external>", line 995, in exec_module
        File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
        File "C:\Users\nolim\AppData\Local\Temp\pip-build-env-vc3vgncr\overlay\Lib\site-packages\setuptools\__init__.py", line 18, in <module>
          from setuptools.extern.six import PY3, string_types
      ModuleNotFoundError: No module named 'setuptools.extern.six'

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 2
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

I could not find much about this error, other that to install setuptools which seems obvious, but even though I have it installed, I cannot get it to work. I assume this is not an modin issue, bit still, I dont know how to proceed.

This is what my pip list looks like:

(.venv) D:\BAAI>pip list Package Version ---------------------------- ----------- absl-py 2.1.0 annotated-types 0.6.0 arch 7.0.0 asttokens 2.4.1 astunparse 1.6.3 blinker 1.8.1 certifi 2024.2.2 charset-normalizer 3.3.2 click 8.1.7 cloudpickle 3.0.0 colorama 0.4.6 comm 0.2.2 contourpy 1.2.1 cycler 0.12.1 dash 2.17.0 dash-core-components 2.0.0 dash-html-components 2.0.0 dash-table 5.0.0 debugpy 1.8.1 decorator 5.1.1 dtw-python 1.4.2 executing 2.0.1 factor_analyzer 0.5.1 filterpy 1.4.5 Flask 3.0.3 flatbuffers 24.3.25 fonttools 4.51.0 fsspec 2024.5.0 gast 0.5.4 gluonts 0.14.4 google-pasta 0.2.0 grpcio 1.63.0 h5py 3.11.0 holidays 0.47 idna 3.7 importlib_metadata 7.1.0 ipykernel 6.29.4 ipython 8.24.0 itsdangerous 2.2.0 jedi 0.19.1 Jinja2 3.1.3 joblib 1.4.0 jupyter_client 8.6.1 jupyter_core 5.7.2 keras 3.3.3 keras-self-attention 0.51.0 kiwisolver 1.4.5 kotsu 0.3.3 lazy_loader 0.4 libclang 18.1.1 llvmlite 0.42.0 Markdown 3.6 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.8.4 matplotlib-inline 0.1.7 mdurl 0.1.2 ml-dtypes 0.3.2 mne 1.7.0 modin 0.30.0 namex 0.0.8 nest-asyncio 1.6.0 numba 0.59.1 numpy 1.26.4 opt-einsum 3.3.0 optree 0.11.0 packaging 24.0 pandas 2.2.2 pandas-flavor 0.6.0 parso 0.8.4 patsy 0.5.6 pillow 10.3.0 pingouin 0.5.4 pip 24.0 platformdirs 4.2.1 plotly 5.22.0 pooch 1.8.1 prompt-toolkit 3.0.43 protobuf 4.25.3 psutil 5.9.8 pure-eval 0.2.2 pyaml 24.4.0 pycatch22 0.4.2 pydantic 2.7.1 pydantic_core 2.18.2 Pygments 2.18.0 pykalman 0.9.7 pyparsing 3.1.2 python-dateutil 2.9.0.post0 pytz 2024.1 pywin32 306 PyYAML 6.0.1 pyzmq 26.0.3 requests 2.31.0 retrying 1.3.4 rich 13.7.1 scikit-base 0.7.7 scikit-learn 1.4.2 scikit-optimize 0.10.1 scikit-posthocs 0.9.0 scipy 1.13.0 seaborn 0.13.2 seasonal 0.3.1 setuptools 69.5.1 six 1.16.0 skpro 2.2.2 sktime 0.29.0 stack-data 0.6.3 statsmodels 0.14.2 tabulate 0.9.0 tenacity 8.2.3 tensorboard 2.16.2 tensorboard-data-server 0.7.2 tensorflow 2.16.1 tensorflow-intel 2.16.1 tensorflow-io-gcs-filesystem 0.31.0 termcolor 2.4.0 threadpoolctl 3.5.0 toolz 0.12.1 tornado 6.4 tqdm 4.66.4 traitlets 5.14.3 tsbootstrap 0.1.1 tslearn 0.6.3 typing_extensions 4.11.0 tzdata 2024.1 urllib3 2.2.1 wcwidth 0.2.13 Werkzeug 3.0.2 wheel 0.43.0 wrapt 1.16.0 xarray 2024.3.0 zipp 3.18.1
YarShev commented 1 month ago

Hi @helloplayer1, thanks for posting this question. What python version are you using? Note that Modin requires python>=3.9.

helloplayer1 commented 1 month ago

I am using Python 3.12.3 in my venv.

YarShev commented 1 month ago

Unfortunately, there is no modin[ray] build for python 3.12 yet but you can install modin[dask] or modin[mpi]. More details in https://github.com/modin-project/modin/issues/7098.

YarShev commented 1 month ago

Let's proceed there. I am going to close this issue as a duplicate of #7098 if you don't mind.