[BUG] Many packages are no longer installable after test command is removed

chrisirhc commented 2 months ago

For those landing on this issue, please see: (thank you @delfick for summarizing this)

This functionality has been deprecated for 5 years, there is a separate issue for discussing if there would have been a better way of removing the functionality [FR] More gradual breakage of setuptools.command.test #4520

For people using pip (or tools using pip, like poetry), PIP_CONSTRAINT set to a file with setuptools<72.0 should work

For people using uv, there seems to be a bug where UV_CONSTRAINT doesn't affect the version of setuptools used in build isolation (see ModuleNotFoundError: No module named 'setuptools.command.test' astral-sh/uv#5551) and in that case the solution is either --no-build-isolation (which should be considered a temporary solution) or getting packages to not use setuptools.command.test or using forks of those packages that comment out the use of setuptools.command.test (until setuptools releases a fix/revert)

To press subscribe to the ticket and/or thumbs up the top post instead of adding "same" comments

Quoted from https://github.com/pypa/setuptools/issues/4519#issuecomment-2255389347

setuptools version

setuptools==72.0.0

Python version

Python 3.9

OS

Linux

Additional environment information

No response

Description

The breakage change released on 72.0 breaks the default build isolation build of many packages since many of these packages do not pin on a particular setuptools version. There is also no way to pin to an older setuptools version as pip doesn't offer a way to do this. Installing setuptools==71 first, then installing the package doesn't work as the default build isolation resolves the dependencies without considering the lock file nor currently installed packages.

Based on these packages need to be patched with setup_requires before install, which is nearly impossible since we'd need to patch all packages.

Expected behavior

Install is successful.

How to Reproduce

Create a virtual environment
Upgrade to latest pip 24.2: python -m pip install --upgrade pip
pip3.9 install doubles==1.4.0

Output

me@my ~/repro-pip-fail
 % pip3.9 install doubles==1.4.0                    
Looking in indexes: http://…
Collecting doubles==1.4.0
  Using cached http://…/doubles-1.4.0.tar.gz (16 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 497, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 2, in <module>
      ModuleNotFoundError: No module named 'setuptools.command.test'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

databius commented 2 months ago

I got a similar error today.

snjtkh commented 2 months ago

we are also getting similar error.

nibe12 commented 2 months ago

i got a similiar error too

vinothsubramanian commented 2 months ago

I am also blocked because of this error

https://github.com/IBM/python-sdk-core/issues/202

smiledelf commented 2 months ago

after an hour of googling, looks like I found the right bug :) this is an issue when installing dbt-core==1.8.4, which lists Logbook==1.5.3 as a requirement.

Latest Logbook (1.7.0) seems to have fixed this but dbt-core still points to the older broken module.

hashken commented 2 months ago

Facing same error here.

09:04:19    ERROR: Command errored out with exit status 1:
09:04:19     command: /var/lib/****/.pex/venvs/s/301ee4ad/venv/bin/python3.9 /var/lib/****/.pex/venvs/s/301ee4ad/venv/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpato8w6eb
09:04:19         cwd: /tmp/pip-download-xmdfaim1/sqlalchemy
09:04:19    Complete output (19 lines):
09:04:19    /tmp/pip-build-env-kra8zma_/overlay/lib/python3.9/site-packages/_distutils_hack/__init__.py:55: UserWarning: Reliance on distutils from stdlib is deprecated. Users must rely on setuptools to provide the distutils module. Avoid importing distutils or import setuptools first, and avoid setting SETUPTOOLS_USE_DISTUTILS=stdlib. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
09:04:19      warnings.warn(
09:04:19    Traceback (most recent call last):
09:04:19      File "/var/lib/****/.pex/venvs/s/301ee4ad/venv/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
09:04:19        main()
09:04:19      File "/var/lib/****/.pex/venvs/s/301ee4ad/venv/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
09:04:19        json_out['return_val'] = hook(**hook_input['kwargs'])
09:04:19      File "/var/lib/****/.pex/venvs/s/301ee4ad/venv/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 114, in get_requires_for_build_wheel
09:04:19        return hook(config_settings)
09:04:19      File "/tmp/pip-build-env-kra8zma_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
09:04:19        return self._get_build_requires(config_settings, requirements=[])
09:04:19      File "/tmp/pip-build-env-kra8zma_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
09:04:19        self.run_setup()
09:04:19      File "/tmp/pip-build-env-kra8zma_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 497, in run_setup
09:04:19        super().run_setup(setup_script=setup_script)
09:04:19      File "/tmp/pip-build-env-kra8zma_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
09:04:19        exec(code, locals())
09:04:19      File "<string>", line 14, in <module>
09:04:19    ModuleNotFoundError: No module named 'setuptools.command.test'

popunbom commented 2 months ago

I was also faced this issue with cssbeautifier library, and solved by installing with --no-build-isolation option in my case.

❯ docker run --rm -it 'python:3.9' /bin/bash

root@81f338564edc:/# python -m pip install --upgrade pip
Requirement already satisfied: pip in /usr/local/lib/python3.9/site-packages (23.0.1)
Collecting pip
  Downloading pip-24.2-py3-none-any.whl (1.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 5.0 MB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.0.1
    Uninstalling pip-23.0.1:
      Successfully uninstalled pip-23.0.1
Successfully installed pip-24.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

root@81f338564edc:/# pip3.9 install cssbeautifier
Collecting cssbeautifier
  Downloading cssbeautifier-1.15.1.tar.gz (25 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-sd2vprw8/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "/tmp/pip-build-env-sd2vprw8/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-sd2vprw8/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 497, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-sd2vprw8/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 9, in <module>
      ModuleNotFoundError: No module named 'setuptools.command.test'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

root@81f338564edc:/# pip3.9 install --no-build-isolation cssbeautifier
Collecting cssbeautifier
  Using cached cssbeautifier-1.15.1.tar.gz (25 kB)
  Preparing metadata (pyproject.toml) ... done
Collecting jsbeautifier (from cssbeautifier)
  Downloading jsbeautifier-1.15.1.tar.gz (75 kB)
  Preparing metadata (pyproject.toml) ... done
Collecting six>=1.13.0 (from cssbeautifier)
  Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting editorconfig>=0.12.2 (from cssbeautifier)
  Downloading EditorConfig-0.12.4.tar.gz (13 kB)
  Preparing metadata (pyproject.toml) ... done
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: cssbeautifier, editorconfig, jsbeautifier
  Building wheel for cssbeautifier (pyproject.toml) ... done
  Created wheel for cssbeautifier: filename=cssbeautifier-1.15.1-py3-none-any.whl size=30536 sha256=fe16be1043cad7c149be1d60d9320b61f2c64ba672681fab279b2a88a71d9ff4
  Stored in directory: /root/.cache/pip/wheels/7f/2d/19/5d9d82d3c7280e2c68a07b906c458f0a5506cb2b812d330383
  Building wheel for editorconfig (pyproject.toml) ... done
  Created wheel for editorconfig: filename=EditorConfig-0.12.4-py3-none-any.whl size=16377 sha256=9acd52f9cb94f6fabd33cc641d4acb07a4714ada4678388b08e45722eabd5607
  Stored in directory: /root/.cache/pip/wheels/0b/7b/d7/1979a68ec6ce660c89044dc57d175e32f1943bcb8937e43f6e
  Building wheel for jsbeautifier (pyproject.toml) ... done
  Created wheel for jsbeautifier: filename=jsbeautifier-1.15.1-py3-none-any.whl size=94721 sha256=c9fefe5c1d947c492e3b8ea0173783df9821f8ca8a46637a8d4a645b9070ae18
  Stored in directory: /root/.cache/pip/wheels/1f/82/d9/63c09b95f4576d2822b61d073de8f4e3b6fd386112dcb9835d
Successfully built cssbeautifier editorconfig jsbeautifier
Installing collected packages: editorconfig, six, jsbeautifier, cssbeautifier
Successfully installed cssbeautifier-1.15.1 editorconfig-0.12.4 jsbeautifier-1.15.1 six-1.16.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.

hauntsaninja commented 2 months ago

Here's how to work around this breakage:

Option 1: Create a file called constraints.txt

setuptools<72

Then set PIP_CONSTRAINT=/path/to/constraints.txt

Option 2: Use --no-build-isolation (and make sure you have an old setuptools installed)

Option 3: Ask the maintainers of each package that is broken like this to adapt to new setuptools and wait for them to make a release

In general, I find it unfortunate that build isolation intends to increase reproducibility, but the tools given to users make this really tricky in practice. I think ^ is pretty hard for most Python users I know to come up with (and lacks e.g. the ability to hash pin). Probably wrong place to rant though, setuptools maintainers probably feel the pain of this more than anyone else! :-)

Edit: if setuptools does decide to revert something here, I have a suggestion in https://github.com/pypa/setuptools/issues/4520 for how the underlying change could be rolled out more gradually

Hannes0730 commented 2 months ago

Here's how to workaround this breakage:

Option 1: Create a file called constraints.txt
setuptools<72
Then set PIP_CONSTRAINT=/path/to/constraints.txt (or UV_CONSTRAINT if using uv)

Option 2: Use --no-build-isolation (and make sure you have an old setuptools installed)

Option 3: Ask the maintainers of each package that is broken like this to adapt to new setuptools and wait for them to make a release

In general, I find it unfortunate that build isolation intends to increase reproducibility, but in practice the tools given to users make this really tricky in practice. I think ^ is pretty hard for most Python users I know to come up with. Probably wrong place to rant though, setuptools maintainers probably feel the pain of this more than anyone else! :-)

Hi, is option 1 compatible with poetry add <package-name>? or is that pip dependent? I tried option 2 but I think --no-build-isolation is not supported in poetry (1.8.2).

RUN poetry install --no-interaction --no-build-isolation --without dev --no-ansi -vvv

I am currently running this on Dockerfile

bebound commented 2 months ago

This is the related PR: https://github.com/pypa/setuptools/pull/4458

anyidea commented 2 months ago

Same issue.

Luke31 commented 2 months ago

Would it be possible to mark this version as yanked in pypi and follow a more gradual approach with deprecation warnings on install as mentioned in https://github.com/pypa/setuptools/issues/4520 ?

Especially installs with poetry seem to have issues and currently I couldn't find a solution to overcome it. Did anybody find a solution for following or similar combination?

docker
poetry==1.7.1
python:3.10
some package not supporting PEP 517 (in our case sqlalchemy==1.4.49 )

EDIT: Found a solution thanks to help of @ilpianista and @Hannes0730 , see my post below: https://github.com/pypa/setuptools/issues/4519#issuecomment-2255390400

(Of course we want to upgrade, but we're still using sqlalchemy==1.4.49 which requires a major upgrade to 2.0 to support PEP 517).

Hannes0730 commented 2 months ago

Here's how to workaround this breakage: Option 1: Create a file called constraints.txt
setuptools<72
Then set PIP_CONSTRAINT=/path/to/constraints.txt (or UV_CONSTRAINT if using uv)

I tried doing Option 1 in the dockerfile, but the error persists.

FROM python:3.12.2-slim as python
ENV PYTHONUNBUFFERED=true
WORKDIR /app

FROM python as poetry
ENV POETRY_HOME=/opt/poetry
ENV POETRY_VIRTUALENVS_IN_PROJECT=true
ENV PATH="$POETRY_HOME/bin:$PATH"
RUN python -c 'from urllib.request import urlopen; print(urlopen("https://install.python-poetry.org").read().decode())' | python -
COPY . ./
ENV PIP_CONSTRAINT=/app/constraints.txt
RUN poetry install --no-interaction --without dev --no-ansi -vvv ---> ERROR: Note: This error originates from the build backend, and is likely not a problem with poetry but with jsbeautifier (1.15.1) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "jsbeautifier (==1.15.1)"

Am I missing something? I tried deleting the docker volume so that it installs all dependencies from scratch.

kndzuedemir commented 2 months ago

We're also facing this issue today. One of our package channels[daphne]==4.0.0 depends setuptools and we got errors below during build;

33.95       Traceback (most recent call last):
33.95         File "/home/python/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
33.95           main()
33.95         File "/home/python/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
33.95           json_out['return_val'] = hook(**hook_input['kwargs'])
33.95                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
33.95         File "/home/python/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
33.95           return hook(config_settings)
33.95                  ^^^^^^^^^^^^^^^^^^^^^
33.95         File "/tmp/pip-build-env-2ppe78wy/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
33.95           return self._get_build_requires(config_settings, requirements=[])
33.95                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
33.95         File "/tmp/pip-build-env-2ppe78wy/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
33.95           self.run_setup()
33.95         File "/tmp/pip-build-env-2ppe78wy/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 497, in run_setup
33.95           super().run_setup(setup_script=setup_script)
33.95         File "/tmp/pip-build-env-2ppe78wy/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 313, in run_setup
33.95           exec(code, locals())
33.95         File "<string>", line 21, in <module>
33.95       NameError: name 'TestCommand' is not defined

ilpianista commented 2 months ago

Especially installs with poetry seem to have issues and currently I couldn't find a solution to overcome it. Did anybody find a solution for following or similar combination?

@Luke31 after several attempts, it looks like this one worked:

$ pip install <problematic dependecy>
$ poetry run pip install <problematic dependecy>
$ poetry install

solarissmoke commented 2 months ago

This completely broke Poetry installation for us, because Poetry enforces build isolation, so you cannot just install your preferred version of setuptools. Ended up with this workaround - exporting the poetry lock file to requirements.txt and then installing with pip:

poetry export -f requirements.txt -o requirements.txt --no-ansi -n --without-hashes
pip install setuptools<72
pip install -r requirements.txt

elliexcoding commented 2 months ago

I tried using --no-build-isolation but my Docker build will fail as it will post ModuleNotFoundError: No module named 'poetry' from dist-packages.

I am using pip install for my requirements.txt.

Edit: For Docker, my fix is to just use 71.1.0 and added to my requirements.txt to get my packages working. autobahn was throwing up errors or me via asyncio.

sandykr01 commented 2 months ago

Good morning, everyone! Yup, ran into a similar problem when our GH workflows started crashing after setuptools dropped this major update. For me, it’s breaking down while installing SQLAlchemy (any solution apart from restricting setuptools to bump up to 72? Why they removed test without any sort of indication):

Collecting SQLAlchemy==1.3.24 (from -r requirements.txt (line 22))
  Downloading SQLAlchemy-1.3.24.tar.gz (6.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.4/6.4 MB 119.3 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-uiic32ek/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-uiic32ek/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-uiic32ek/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 497, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-uiic32ek/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 14, in <module>
      ModuleNotFoundError: No module named 'setuptools.command.test'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

Artui commented 2 months ago

Issue for us as well. Persists even though setuptools are pinned to 71.1.0.

elliexcoding commented 2 months ago

Issue for us as well. Persists even though setuptools are pinned to 71.1.0.

Have you tried running a pip install via requirements.txt rather than poetry.lock/pyproject.toml? That was what got it working for me.

ilpianista commented 2 months ago

Issue for us as well. Persists even though setuptools are pinned to 71.1.0.

@Artui have a look to https://github.com/pypa/setuptools/issues/4519#issuecomment-2255263220

hashken commented 2 months ago

For our pex builds, the pip constraint approach outlined in https://github.com/pypa/setuptools/issues/4519#issuecomment-2254983472 solves the issue

echo "setuptools<72" > /tmp/constraint.txt
EXPORT PIP_CONSTRAINT = /tmp/constraint.txt
<run the pex command>

agusmakmun commented 2 months ago

In my case, we disable some packages that causing this issue, for example:

# elastic-apm==6.6.0  # https://github.com/elastic/apm-agent-python
# djlint==1.34.1  # https://github.com/Riverside-Healthcare/djLint

But seems celery still have that: https://github.com/celery/celery/blob/main/setup.py#L7

AmosRosmarin commented 2 months ago

pip install -I --force-reinstall setuptools==71.0.0

works for me

simonwhitaker commented 2 months ago

Here's a repo demonstrating how you can work around this in your Github Actions.

https://github.com/simonwhitaker/setuptools-demo

Artui commented 2 months ago

Issue for us as well. Persists even though setuptools are pinned to 71.1.0.

@Artui have a look to #4519 (comment)

requirements approach works, but we run poetry everywhere, would be nice to have a workaround for that too.

Was that command deprecated first at some point? Or was it removed right away? Feels like the release should mark it as deprecated first instead of ripping it out...

Hannes0730 commented 2 months ago

Quote reply

FOR POETRY I think this worked for me with PIP_CONSTRAINT combined.

FROM python:3.12.2-slim as python
ENV PYTHONUNBUFFERED=true
WORKDIR /app

FROM python as poetry
ENV POETRY_HOME=/opt/poetry
ENV POETRY_VIRTUALENVS_IN_PROJECT=true
ENV PATH="$POETRY_HOME/bin:$PATH"
RUN python -c 'from urllib.request import urlopen; print(urlopen("https://install.python-poetry.org").read().decode())' | python -
COPY . ./

# Install litestar because of broken setuptools unable to install jsbeautifier
ENV PIP_CONSTRAINT=/app/constraints.txt
RUN pip install litestar["standard"]
RUN poetry run pip install litestar["standard"]

RUN poetry install --no-interaction --without dev --no-ansi -vvv

AfikGrinstein commented 2 months ago

Can we revert this PR https://github.com/pypa/setuptools/pull/4458 ? I tried everything and was still not able to get it to work. We are working with poetry.

bissli82 commented 2 months ago

Same here - please revert :(

illia-khorunzhyi-metro commented 2 months ago

With this work for me with Docker, python 3.12

RUN pip3 install -I --force-reinstall setuptools==71.0.0
RUN pip3 install --no-build-isolation --no-cache-dir -r requirements.txt

Hannes0730 commented 2 months ago

AfikGrinstein

Hi, have you tried https://github.com/pypa/setuptools/issues/4519#issuecomment-2255337022 ? I also use poetry and it worked.

AfikGrinstein commented 2 months ago

AfikGrinstein

Hi, have you tried #4519 (comment) ? I also use poetry and it worked.

Yes. Didn't work for me. :-(

petrprikryl commented 2 months ago

Same here - please revert :(

+1

samzong commented 2 months ago

Same issue with me.

apakhomov commented 2 months ago

Revert plz, this breaks lots of projects

bjgrosse commented 2 months ago

This worked for us using poetry and docker. In the dockerfile:

WORKDIR /code
# these two new lines are the fix
RUN echo "setuptools<72" > "constraints.txt"
ENV PIP_CONSTRAINT=/code/constraints.txt

RUN poetry install --no-interaction --no-ansi

yitian-reevo commented 2 months ago

same here +1, with Nylas sdk

delfick commented 2 months ago

@chrisirhc are you able to please add to the description of the ticket that

This functionality has been deprecated for 5 years, there is a separate issue for discussing if there would have been a better way of removing the functionality https://github.com/pypa/setuptools/issues/4520
For people using pip (or tools using pip, like poetry), PIP_CONSTRAINT set to a file with setuptools<72.0 should work
For people using uv, there seems to be a bug where UV_CONSTRAINT doesn't affect the version of setuptools used in build isolation (see https://github.com/astral-sh/uv/issues/5551) and in that case the solution is either --no-build-isolation (which should be considered a temporary solution) or getting packages to not use setuptools.command.test or using forks of those packages that comment out the use of setuptools.command.test (until setuptools releases a fix/revert)
To press subscribe to the ticket and/or thumbs up the top post instead of adding "same" comments

:)

edit: I've since suggested a couple more dotpoints https://github.com/pypa/setuptools/issues/4519#issuecomment-2255690881

edit2: the version has been yanked https://github.com/pypa/setuptools/issues/4519#issuecomment-2255940870

Luke31 commented 2 months ago

@ilpianista @Hannes0730 Thanks for your help, following worked for me too in poetry:

RUN echo "setuptools<72" > /code/constraints.txt
ENV PIP_CONSTRAINT=/code/constraints.txt
RUN poetry run pip install sqlalchemy==1.4.49
RUN poetry install

(sqlalchemy==1.4.49 is the problematic package here not supporting PEP 517) The current workaround for poetry seems to be... not using poetry to install 👌 (Somehow RUN pip install sqlalchemy==1.4.49 wasn't required but that might be due to still using python 3.10 🤔 )

halbow commented 2 months ago

FYI for people using pip-tools that wants a workaround, make sure you both install setuptools version 71.1.0 and use pip-sync and pip-compile with the --pip-args --no-build-isolation flag

pip install --upgrade pip-tools pip wheel;
pip install setuptools==71.1.0;
pip-sync requirements.txt --pip-args --no-build-isolation

Artui commented 2 months ago

This worked for us using poetry and docker. In the dockerfile:

WORKDIR /code
# these two new lines are the fix
RUN echo "setuptools<72" > "constraints.txt"
ENV PIP_CONSTRAINT=/code/constraints.txt

RUN poetry install --no-interaction --no-ansi

Doesn't seem to work for us... I've no idea what's going on "under the hood" here, but it seems to me that poetry is not respecting the setuptools of the virtualenv (we're using python:3.12.4-slim here)

stamparm commented 2 months ago

seems that this one is going out of the control, based on quick Github search. it would be really nice if somebody from the dev team would make a definite statement whether we should do "dirty patches" in our deployment scripts with some random workarounds OR we should wait for the resolution in the setuptools itself. in our case, we have more than a dozen of projects with custom deployment scripts and complex inter-dependencies.

currently closely watching for this issue/thread to decide on future steps

CesarFalco commented 2 months ago

same here +1

bissli82 commented 2 months ago

@ilpianista @Hannes0730 Thanks for your help, following worked for me too in poetry:
RUN echo "setuptools<72" > /code/constraints.txt
ENV PIP_CONSTRAINT=/code/constraints.txt
RUN poetry run pip install sqlalchemy==1.4.49
RUN poetry install
The current workaround for potery seems to be... not using poetry to install 👌 (Somehow RUN pip install sqlalchemy==1.4.49 wasn't required but that might be due to still using python 3.10 🤔 )

This workaround seems to be working, install the problematic package via pip(with constraints) and the rest via poetry

eyareyar123 commented 2 months ago

Same issue here +1

nakamichiworks commented 2 months ago

For poetry users, exporting requirements.txt and pip install with the constraint worked for me.

export PIP_CONSTRAINT=constraints.txt
echo "setuptools<72" > constraints.txt
poetry export -o requirements.txt
pip install -r requirements.txt

ferrarijefferson commented 2 months ago

For uv users, this worked for me:

uv venv
source .venv/bin/activate
uv pip install setuptools==71.1.0 poetry==1.8.3
uv pip install --no-build-isolation -r requirements.txt

jamesbuddrige commented 2 months ago

Also unable to use poetry at the moment...

pfaion commented 2 months ago

This breaks ~10 packages in my dependency chain which just specify:

[build-system]
requires = ["setuptools"]

so they don't pin their setuptools build dependency, which makes them pull the latest version on every build. And that build happens in the isolated build environments for each package, so I don't even really directly affect that somehow.

Not a good start into the week.

Makes me wonder if you should generally pin your build environment, there shouldn't be any harm since it does not conflict with anything else? Assuming that you use build isolation though.

omerbe-nym commented 2 months ago

(https://github.com/pypa/setuptools/issues/4519#issuecomment-2255390400)

Thank you, this solved our issue (using Poetry 1.8.3, Python 3.11.4).

RUN echo "setuptools<72" > constraints.txt

ENV PIP_CONSTRAINT=constraints.txt

RUN poetry config virtualenvs.create false && \
    poetry run pip install sqlalchemy==1.3.24 && \
    poetry install --no-cache && \

I could not understand why when I degraded the setuptools to version below 72.0.0 it kept cruising with the error of ModuleNotFoundError: No module named 'setuptools.command.test'.

pypa / setuptools