search

ray-project / ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
34.16k stars 5.8k forks source link

[core] Failed to build Ray from source: missing pip #42505

Open GithubZhitao opened 10 months ago

GithubZhitao commented 10 months ago

What happened + What you expected to happen

I follow the building on MacOs process to build from source. When I exec "pip install -e . --verbose"，the errors happened like this：

Building wheels for collected packages: ray
  Running command Building editable for ray (pyproject.toml)
  running editable_wheel
  creating /private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-wheel-wat31rkr/.tmp-17nip05a/ray.egg-info
  writing /private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-wheel-wat31rkr/.tmp-17nip05a/ray.egg-info/PKG-INFO
  writing dependency_links to /private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-wheel-wat31rkr/.tmp-17nip05a/ray.egg-info/dependency_links.txt
  writing entry points to /private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-wheel-wat31rkr/.tmp-17nip05a/ray.egg-info/entry_points.txt
  writing requirements to /private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-wheel-wat31rkr/.tmp-17nip05a/ray.egg-info/requires.txt
  writing top-level names to /private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-wheel-wat31rkr/.tmp-17nip05a/ray.egg-info/top_level.txt
  writing manifest file '/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-wheel-wat31rkr/.tmp-17nip05a/ray.egg-info/SOURCES.txt'
  reading manifest file '/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-wheel-wat31rkr/.tmp-17nip05a/ray.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file '/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-wheel-wat31rkr/.tmp-17nip05a/ray.egg-info/SOURCES.txt'
  creating '/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-wheel-wat31rkr/.tmp-17nip05a/ray-2.9.0.dist-info'
  creating /private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-wheel-wat31rkr/.tmp-17nip05a/ray-2.9.0.dist-info/WHEEL
  running build_py
  running build_ext
  /Users/xxxxx/development/ray/ray-ray-2.9.0/venv/bin/python: No module named pip
  Traceback (most recent call last):
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 156, in run
      self._create_wheel_file(bdist_wheel)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 345, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 268, in _run_build_commands
      self._run_build_subcommands()
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 295, in _run_build_subcommands
      self.run_command(name)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 963, in run_command
      super().run_command(command)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "<string>", line 745, in run
    File "<string>", line 649, in pip_run
    File "<string>", line 525, in build
    File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 374, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['/Users/xxxx/development/ray/ray-ray-2.9.0/venv/bin/python', '-m', 'pip', 'install', '-q', '--target=/Users/xxxx/development/ray/ray-ray-2.9.0/python/ray/thirdparty_files', 'psutil', 'setproctitle==1.2.2', 'colorama']' returned non-zero exit status 1.
  /private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py:988: _DebuggingTips: Problem in editable installation.
  !!

          ********************************************************************************
          An error happened while installing `ray` in editable mode.

          The following steps are recommended to help debug this problem:

          - Try to install the project normally, without using the editable mode.
            Does the error still persist?
            (If it does, try fixing the problem before attempting the editable mode).
          - If you are using binary extensions, make sure you have all OS-level
            dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
          - Try the latest version of setuptools (maybe the error was already fixed).
          - If you (or your project dependencies) are using any setuptools extension
            or customization, make sure they support the editable mode.

          After following the steps above, if the problem still persists and
          you think this is related to how setuptools handles editable installations,
          please submit a reproducible example
          (see https://stackoverflow.com/help/minimal-reproducible-example) to:

              https://github.com/pypa/setuptools/issues

          See https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.
          ********************************************************************************

  !!
    cmd_obj.run()
  Traceback (most recent call last):
    File "/Users/xxxx/development/ray/ray-ray-2.9.0/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/Users/xxxx/development/ray/ray-ray-2.9.0/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/xxxx/development/ray/ray-ray-2.9.0/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 273, in build_editable
      return hook(wheel_directory, config_settings, metadata_directory)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 436, in build_editable
      return self._build_with_temp_dir(
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 389, in _build_with_temp_dir
      self.run_setup()
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 480, in run_setup
      super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 757, in <module>
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 103, in setup
      return distutils.core.setup(**attrs)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 963, in run_command
      super().run_command(command)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 156, in run
      self._create_wheel_file(bdist_wheel)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 345, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 268, in _run_build_commands
      self._run_build_subcommands()
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 295, in _run_build_subcommands
      self.run_command(name)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
      self.distribution.run_command(command)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 963, in run_command
      super().run_command(command)
    File "/private/var/folders/8g/3vvjhtzs7dsf5kktm0zzqyt40000gn/T/pip-build-env-r8gurbm5/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "<string>", line 745, in run
    File "<string>", line 649, in pip_run
    File "<string>", line 525, in build
    File "/usr/local/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 374, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['/Users/xxxx/development/ray/ray-ray-2.9.0/venv/bin/python', '-m', 'pip', 'install', '-q', '--target=/Users/xxxx/development/ray/ray-ray-2.9.0/python/ray/thirdparty_files', 'psutil', 'setproctitle==1.2.2', 'colorama']' returned non-zero exit status 1.
  error: subprocess-exited-with-error

  × Building editable for ray (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.

### Versions / Dependencies

Ray 2.9.0 source code

Python 3.9.5 (default, May  4 2021, 03:29:30) 
[Clang 11.0.0 (clang-1100.0.33.17)] on darwin

Macos Mojave 10.14.6

Reproduction script

# Install Ray.
cd python/
# You may need to set the following two env vars if your platform is MacOS ARM64(M1).
# See https://github.com/grpc/grpc/issues/25082 for more details.
# export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
# export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1
pip install -e . --verbose  # Add --user if you see a permission denied error.

Issue Severity

High: It blocks me from completing my task.

simonsays1980 commented 7 months ago

Hey @anyscalesam I don't think this is something we can change in RLlib. It looks to me like a more general bug is happening during installation. This is why I removed the label rllib. Do you think this is something that can be solved by us? Let's discuss!

peytondmurray commented 4 months ago

I think I'm seeing the exact same thing on Linux as well. Before building I ran git clean -fdx ./ to clear out all preexisting build artifacts, and then I followed the docs:

  1. cd python/ray/dashboard/client
  2. npm ci
  3. npm run build
  4. cd ../../..
  5. pip install -ve .

the following build error happens:

Using pip 24.1.2 from /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages/pip (python 3.11)
Obtaining file:///home/pdmurray/Desktop/workspace/ray/python
  Running command pip subprocess to install build dependencies
  Using pip 24.1.2 from /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages/pip (python 3.11)
  Collecting setuptools>=40.8.0
    Obtaining dependency information for setuptools>=40.8.0 from https://files.pythonhosted.org/packages/51/a0/ee460cc54e68afcf33190d198299c9579a5eafeadef0016ae8563237ccb6/setuptools-71.1.0-py3-none-any.whl.metadata
    Using cached setuptools-71.1.0-py3-none-any.whl.metadata (6.6 kB)
  Using cached setuptools-71.1.0-py3-none-any.whl (2.3 MB)
  Installing collected packages: setuptools
  Successfully installed setuptools-71.1.0
  Installing build dependencies ... done
  Running command Checking if build backend supports build_editable
  Checking if build backend supports build_editable ... done
  Running command Getting requirements to build editable
  Getting requirements to build editable ... done
  Running command pip subprocess to install backend dependencies
  Using pip 24.1.2 from /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages/pip (python 3.11)
  Collecting cython>=0.29.32
    Obtaining dependency information for cython>=0.29.32 from https://files.pythonhosted.org/packages/45/82/077c13035d6f45d8b8b74d67e9f73f2bfc54ef8d1f79572790f6f7d2b4f5/Cython-3.0.10-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
    Using cached Cython-3.0.10-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB)
  Collecting wheel
    Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl.metadata
    Using cached wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB)
  Using cached Cython-3.0.10-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
  Using cached wheel-0.43.0-py3-none-any.whl (65 kB)
  Installing collected packages: wheel, cython
    Creating /tmp/pip-build-env-8z35utic/normal/bin
    changing mode of /tmp/pip-build-env-8z35utic/normal/bin/wheel to 755
    changing mode of /tmp/pip-build-env-8z35utic/normal/bin/cygdb to 755
    changing mode of /tmp/pip-build-env-8z35utic/normal/bin/cython to 755
    changing mode of /tmp/pip-build-env-8z35utic/normal/bin/cythonize to 755
  Successfully installed cython-3.0.10 wheel-0.43.0
  Installing backend dependencies ... done
  Running command Preparing editable metadata (pyproject.toml)
  running dist_info
  creating /tmp/pip-modern-metadata-n05qqvil/ray.egg-info
  writing /tmp/pip-modern-metadata-n05qqvil/ray.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-modern-metadata-n05qqvil/ray.egg-info/dependency_links.txt
  writing entry points to /tmp/pip-modern-metadata-n05qqvil/ray.egg-info/entry_points.txt
  writing requirements to /tmp/pip-modern-metadata-n05qqvil/ray.egg-info/requires.txt
  writing top-level names to /tmp/pip-modern-metadata-n05qqvil/ray.egg-info/top_level.txt
  writing manifest file '/tmp/pip-modern-metadata-n05qqvil/ray.egg-info/SOURCES.txt'
  reading manifest file '/tmp/pip-modern-metadata-n05qqvil/ray.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file '/tmp/pip-modern-metadata-n05qqvil/ray.egg-info/SOURCES.txt'
  creating '/tmp/pip-modern-metadata-n05qqvil/ray-3.0.0.dev0.dist-info'
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: click>=7.0 in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from ray==3.0.0.dev0) (8.1.7)
Requirement already satisfied: filelock in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from ray==3.0.0.dev0) (3.15.4)
Requirement already satisfied: jsonschema in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from ray==3.0.0.dev0) (4.23.0)
Requirement already satisfied: msgpack<2.0.0,>=1.0.0 in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from ray==3.0.0.dev0) (1.0.8)
Requirement already satisfied: packaging in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from ray==3.0.0.dev0) (24.1)
Requirement already satisfied: protobuf!=3.19.5,>=3.15.3 in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from ray==3.0.0.dev0) (4.25.4)
Requirement already satisfied: pyyaml in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from ray==3.0.0.dev0) (6.0.1)
Requirement already satisfied: aiosignal in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from ray==3.0.0.dev0) (1.3.1)
Requirement already satisfied: frozenlist in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from ray==3.0.0.dev0) (1.4.1)
Requirement already satisfied: requests in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from ray==3.0.0.dev0) (2.32.3)
Requirement already satisfied: attrs>=22.2.0 in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from jsonschema->ray==3.0.0.dev0) (23.2.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from jsonschema->ray==3.0.0.dev0) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from jsonschema->ray==3.0.0.dev0) (0.35.1)
Requirement already satisfied: rpds-py>=0.7.1 in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from jsonschema->ray==3.0.0.dev0) (0.19.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from requests->ray==3.0.0.dev0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from requests->ray==3.0.0.dev0) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from requests->ray==3.0.0.dev0) (2.2.2)
Requirement already satisfied: certifi>=2017.4.17 in /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages (from requests->ray==3.0.0.dev0) (2024.7.4)
Building wheels for collected packages: ray
  Running command Building editable for ray (pyproject.toml)
  running editable_wheel
  creating /tmp/pip-wheel-47pmnapg/.tmp-p2qwv_z9/ray.egg-info
  writing /tmp/pip-wheel-47pmnapg/.tmp-p2qwv_z9/ray.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-wheel-47pmnapg/.tmp-p2qwv_z9/ray.egg-info/dependency_links.txt
  writing entry points to /tmp/pip-wheel-47pmnapg/.tmp-p2qwv_z9/ray.egg-info/entry_points.txt
  writing requirements to /tmp/pip-wheel-47pmnapg/.tmp-p2qwv_z9/ray.egg-info/requires.txt
  writing top-level names to /tmp/pip-wheel-47pmnapg/.tmp-p2qwv_z9/ray.egg-info/top_level.txt
  writing manifest file '/tmp/pip-wheel-47pmnapg/.tmp-p2qwv_z9/ray.egg-info/SOURCES.txt'
  reading manifest file '/tmp/pip-wheel-47pmnapg/.tmp-p2qwv_z9/ray.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file '/tmp/pip-wheel-47pmnapg/.tmp-p2qwv_z9/ray.egg-info/SOURCES.txt'
  creating '/tmp/pip-wheel-47pmnapg/.tmp-p2qwv_z9/ray-3.0.0.dev0.dist-info'
  creating /tmp/pip-wheel-47pmnapg/.tmp-p2qwv_z9/ray-3.0.0.dev0.dist-info/WHEEL
  running build_py
  running build_ext
  /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/bin/python3.11: No module named pip
  Traceback (most recent call last):
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 153, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 355, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 278, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 305, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 956, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
      cmd_obj.run()
    File "<string>", line 749, in run
    File "<string>", line 651, in pip_run
    File "<string>", line 519, in build
    File "/home/pdmurray/.pyenv/versions/3.11.9/lib/python3.11/subprocess.py", line 413, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['/home/pdmurray/.pyenv/versions/3.11.9/envs/ray/bin/python3.11', '-m', 'pip', 'install', '-q', '--target=/home/pdmurray/Desktop/workspace/ray/python/ray/thirdparty_files', 'psutil', 'setproctitle==1.2.2', 'colorama']' returned non-zero exit status 1.
  /tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py:989: _DebuggingTips: Problem in editable installation.
  !!

          ********************************************************************************
          An error happened while installing `ray` in editable mode.

          The following steps are recommended to help debug this problem:

          - Try to install the project normally, without using the editable mode.
            Does the error still persist?
            (If it does, try fixing the problem before attempting the editable mode).
          - If you are using binary extensions, make sure you have all OS-level
            dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
          - Try the latest version of setuptools (maybe the error was already fixed).
          - If you (or your project dependencies) are using any setuptools extension
            or customization, make sure they support the editable mode.

          After following the steps above, if the problem still persists and
          you think this is related to how setuptools handles editable installations,
          please submit a reproducible example
          (see https://stackoverflow.com/help/minimal-reproducible-example) to:

              https://github.com/pypa/setuptools/issues

          See https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.
          ********************************************************************************

  !!
    cmd_obj.run()
  Traceback (most recent call last):
    File "/home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 273, in build_editable
      return hook(wheel_directory, config_settings, metadata_directory)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 453, in build_editable
      return self._build_with_temp_dir(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 397, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 497, in run_setup
      super().run_setup(setup_script=setup_script)
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 313, in run_setup
      exec(code, locals())
    File "<string>", line 761, in <module>
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 108, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 184, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
      dist.run_commands()
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 970, in run_commands
      self.run_command(cmd)
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 956, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 153, in run
      self._create_wheel_file(bdist_wheel)
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 355, in _create_wheel_file
      files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 278, in _run_build_commands
      self._run_build_subcommands()
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/command/editable_wheel.py", line 305, in _run_build_subcommands
      self.run_command(name)
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
      self.distribution.run_command(command)
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 956, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-8z35utic/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
      cmd_obj.run()
    File "<string>", line 749, in run
    File "<string>", line 651, in pip_run
    File "<string>", line 519, in build
    File "/home/pdmurray/.pyenv/versions/3.11.9/lib/python3.11/subprocess.py", line 413, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['/home/pdmurray/.pyenv/versions/3.11.9/envs/ray/bin/python3.11', '-m', 'pip', 'install', '-q', '--target=/home/pdmurray/Desktop/workspace/ray/python/ray/thirdparty_files', 'psutil', 'setproctitle==1.2.2', 'colorama']' returned non-zero exit status 1.
  error: subprocess-exited-with-error

  × Building editable for ray (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/bin/python3.11 /home/pdmurray/.pyenv/versions/3.11.9/envs/ray/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_editable /tmp/tmpo84pfrgb
  cwd: /home/pdmurray/Desktop/workspace/ray/python
  Building editable for ray (pyproject.toml) ... error
  ERROR: Failed building editable for ray
Failed to build ray
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (ray)

System Info

python -VV: Python 3.11.9 (main, Jul 5 2024, 12:36:19) [GCC 14.1.1 20240522] uname -rms: Linux 6.9.10-arch1-1 x86_64 git rev-parse HEAD: ff789b95b8bd22af257fc168a9cc8ab689af6ede

peytondmurray commented 4 months ago

Interestingly, it seems like python setup.py build_ext (deprecated! probably not a good idea to do this!) actually proceeds to download vendored packages to thirdparty_files as intended.

mattip commented 1 week ago

This may be a documentation problem. The python environment is missing pip. I don't known how you get a virtual environment without pip, that seems wrong. Can you check on the forums for the tool you used to create this, maybe pyenv?

/home/pdmurray/.pyenv/versions/3.11.9/envs/ray/bin/python3.11: No module named pip