[BUG] Many packages are no longer installable after test command is removed

[chrisirhc]

For those landing on this issue, please see: (thank you @delfick for summarizing this)

• This functionality has been deprecated for 5 years, there is a separate issue for discussing if there would have been a better way of removing the functionality [FR] More gradual breakage of setuptools.command.test #4520
• For people using pip (or tools using pip, like poetry), PIP_CONSTRAINT set to a file with setuptools<72.0 should work
• For people using uv, there seems to be a bug where UV_CONSTRAINT doesn't affect the version of setuptools used in build isolation (see ModuleNotFoundError: No module named 'setuptools.command.test' astral-sh/uv#5551) and in that case the solution is either --no-build-isolation (which should be considered a temporary solution) or getting packages to not use setuptools.command.test or using forks of those packages that comment out the use of setuptools.command.test (until setuptools releases a fix/revert)
• To press subscribe to the ticket and/or thumbs up the top post instead of adding "same" comments

Quoted from https://github.com/pypa/setuptools/issues/4519#issuecomment-2255389347

---

setuptools version

setuptools==72.0.0

Python version

Python 3.9

OS

Linux

Additional environment information

No response

Description

The breakage change released on 72.0 breaks the default build isolation build of many packages since many of these packages do not pin on a particular setuptools version. There is also no way to pin to an older setuptools version as pip doesn't offer a way to do this. Installing setuptools==71 first, then installing the package doesn't work as the default build isolation resolves the dependencies without considering the lock file nor currently installed packages.

Based on these packages need to be patched with setup_requires before install, which is nearly impossible since we'd need to patch all packages.

Expected behavior

Install is successful.

How to Reproduce

1. Create a virtual environment
2. Upgrade to latest pip 24.2: python -m pip install --upgrade pip
3. pip3.9 install doubles==1.4.0

Output

me@my ~/repro-pip-fail
 % pip3.9 install doubles==1.4.0                    
Looking in indexes: http://…
Collecting doubles==1.4.0
  Using cached http://…/doubles-1.4.0.tar.gz (16 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 497, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 2, in <module>
      ModuleNotFoundError: No module named 'setuptools.command.test'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[apakhomov]

Guys, advising "downgrade setuptools, downgrade pip" and putting minuses under revert proposals, it's easy to do when you develop single project on your own pc in your own venv. When you have tens or hundreds of projects, that use dependencies of dependencies, the game becomes much more exciting. rebuild base images, rebuild depending images, rebuild libraries... Do you really understand the scope of work?

[amine-el-amrani]

Same issue here +1

[younsihamza]

I was facing this problem when I wanted to install Django Channels with Daphne. I found this solution and used the following command: pip3 install --no-build-isolation --no-cache-dir "setuptools<72"

[solebox]

thanks @omerbe-nym worked for us

[justinclift]

@jamesbuddrige Try the stuff in https://github.com/pypa/setuptools/issues/4519#issuecomment-2255446798/

[andrewokello]

For those using docker and poetry,

This is how my dockerfile looks now, and it works for me

# ---- Base Python ----
FROM python:3.11.4-slim AS base

# Create app directory
WORKDIR /app

# ---- Dependencies ----
FROM base AS dependencies

# Install pip and Poetry
RUN pip install --upgrade pip && \
    pip install poetry

# Copy poetry.lock* in case it doesn't exist in the repo
COPY ./poetry.lock ./pyproject.toml /app/

# Install project dependencies.
RUN export PIP_CONSTRAINT=constraints.txt
RUN echo "setuptools<72" > constraints.txt
RUN poetry export -o requirements.txt
RUN pip install setuptools==71.1.0
RUN pip install --no-build-isolation -r requirements.txt

# ---- Copy Files/Build ----
FROM dependencies AS build

WORKDIR /app

COPY . /app

# --- Release ----
FROM base AS release

COPY --from=dependencies /usr/local/bin /usr/local/bin
COPY --from=dependencies /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY --from=build /app /app

EXPOSE 8000

# Setup entrypoint
COPY ./docker/entrypoint.sh /app

ENTRYPOINT ["sh", "/app/entrypoint.sh"]
CMD ["gunicorn", "config.wsgi:application", "--bind", "0.0.0.0:8000"]

[LewisCowlesMotive]

It'd be great to know when this is solved so folks can upgrade again. setuptools wasn't even an explicit dependency and now it is an explicit devdependency.

[berkaycamur]

Guys I'm facing with an issue during installation with poetry on docker and in first it says ModuleNotFoundError: No module named 'setuptools.command.test' and then it returns error about a package which is trying to be installed. Is that problem also depends on that error?

[omerbe-nym]

Guys I'm facing with an issue during installation with poetry on docker and in first it says ModuleNotFoundError: No module named 'setuptools.command.test' and then it returns error about a package which is trying to be installed. Is that problem also depends on that error?

@berkaycamur Try to look at https://github.com/pypa/setuptools/issues/4519#issuecomment-2255446798

[jamesbuddrige]

@jamesbuddrige Try the stuff in #4519 (comment)

We're working across 50+ repo's so this isn't an ideal solution, but will give it a go. @justinclift

[JarroVGIT]

after an hour of googling, looks like I found the right bug :) this is an issue when installing dbt-core==1.8.4, which lists Logbook==1.5.3 as a requirement.

Latest Logbook (1.7.0) seems to have fixed this but dbt-core still points to the older broken module.

For some reason, this seems to work fine now with us:

python -m pip install --force-reinstall setuptools==71.1.0 && python -m pip install --no-cache-dir --no-build-isolation dbt-snowflake==1.7.2

I don't know man

[agusmakmun]

@jaraco can you please roll back the latest version? As it impacting many packages. After that, we can do the deprecate solution to ensure old packages still supported.

[petrprikryl]

Could it 72.0.0 marked as yanked before other essential tools will be working with this change? :pray:

[llybin]

pipenv

Pipfile

[packages]
setuptools = "==71.1.0"  # sse docker/Dockerfile:54

pipenv lock

Dockerfile

# https://github.com/pypa/setuptools/issues/4519#issuecomment-2255390400
# try to remove these lines after the issue is resolved
# remove setuptools from Pipfile
# also perhaps need wait for when other packages will be updated with compatible setuptools
RUN echo "setuptools<72" > /app/constraints.txt
ENV PIP_CONSTRAINT=/app/constraints.txt
RUN pip install --no-build-isolation --no-cache-dir "setuptools<72"

[gmyrianthous]

Yank v72.0.0 ASAP!

[royire]

Yank, Yank, Yank!!!

[elliottrabac]

after an hour of googling, looks like I found the right bug :) this is an issue when installing dbt-core==1.8.4, which lists Logbook==1.5.3 as a requirement. Latest Logbook (1.7.0) seems to have fixed this but dbt-core still points to the older broken module.

For some reason, this seems to work fine now with us:

python -m pip install --force-reinstall setuptools==71.1.0 && python -m pip install --no-cache-dir --no-build-isolation dbt-snowflake==1.7.2

I don't know man

Works for us as well to fix the dbt package issue 👍

[djuarezg]

Did someone figure out how to fix this on Pants package builds?

[tisnik]

For anybody using PDM, there's our solution - not nice, but we have the need to pass our CI ASAP:

        # force install older setuptools version
    pip install -I --force-reinstall setuptools==71.0.0
    pip show setuptools
    export PIP_CONSTRAINT=constraints.txt
    pdm install --no-isolation --dev -v

(--dev is not needed if you want to install non-dev dependencies, and -v is just to increase verbosity level)

[berkaycamur]

Guys in brief, the main solution is downgrade the version of the setuptools ( 69.0.2 worked fine on me ). There is bunch of solutions above but in my case, setuptools was preventing me to install a package using poetry and #4519 (comment) worked for me

[villoro]

I also had problems with this. In my case I need to install the packages in a github action and in docker with poetry. It's a DBT project, so as mentioned in https://github.com/pypa/setuptools/issues/4519#issuecomment-2254942140 it's because of logbook. So here is how I fixed it

1. Constraints

Create the constraints.txt with just setuptools<72

2. Docker

COPY constraints.txt .
ENV PIP_CONSTRAINT=constraints.txt

RUN poetry run pip install logbook==1.5.3 && \
    poetry install --no-cache

Replace logbook==1.5.3 with whatever package is giving you problems

3. Github action

    - name: Handle nasty bug # https://github.com/pypa/setuptools/issues/4519#issuecomment-2255390400
      run: |
        echo "PIP_CONSTRAINT=constraints.txt" >> $GITHUB_ENV
        poetry run pip install logbook==1.5.3

    - name: Install dependencies
      run: poetry install --no-cache

    # I had some problems with `pre-commit`, so I had to remove the env var afterwards:
    - name: Remove PIP_CONSTRAINT
      run: echo "PIP_CONSTRAINT=" >> $GITHUB_ENV

Replace logbook==1.5.3 with whatever package is giving you problems

Hope it helps!

[RubTalha]

https://stackoverflow.com/a/78806669/26556668

[LewisCowlesMotive]

https://github.com/pypa/setuptools/issues/4519#issuecomment-2254983472 does not work for folks who use poetry.

[Afoucaul]

For Poetry/Docker users, I've managed to contain the fix to these three steps, though I did have to iterate to find all the dependencies that were broken

RUN mkdir -p /opt/pip && echo "setuptools<72" >/opt/pip/constraints.txt
ENV PIP_CONSTRAINT=/opt/pip/constraints.txt
RUN \
  mkdir /build/ \
  && poetry export --without-hashes \
    | grep \
      -e intervaltree \
      -e vcrpy \
      # -e any_other_package
    >/build/requirements.setuptools-bugged.txt \
  && poetry run pip install -r /build/requirements.setuptools-bugged.txt

You can run that before your usual RUN poetry install steps, which won't re-install those packages

[hvdklauw]

tbh.. yes the workarounds work, but defining packages (versioned) outside of your normal tools just to get them installed is just not a workable solution for any kind of production app.

[pcreux]

[mwPandoraid]

please yank this, none of these workarounds should be necessary and some of them don't even work - disabling build isolation results in build errors for me.

i understand the desire to remove something that's obsolete but clearly this wasn't the right moment or way to do it.

[deceze]

This worked for me for Poetry without picking out individual broken packages:

poetry export -o requirements.txt --without-hashes
poetry export -f constraints.txt -o constraints.txt --without-hashes
PIP_CONSTRAINT="$(pwd)/constraints.txt" poetry run pip -r requirements.txt

In other words, let poetry export requirements and constraints to files, and make sure you use the same options as you'd use for poetry install (e.g. --only main). You may or may not need --without-hashes, depending on where you're getting what dependencies from. Try omitting --without-hashes, it's probably better to use hashes if it works for you. Finally let pip install the dependencies from requirements.txt and obeying the constraints. This assumes you do have setuptools<72 locked in Poetry, otherwise the constraints won't do much.

[gmyrianthous]

Any workarounds for poetry ?

setuptools = "<72.0"

doesn't seem to work for some reason.

PS: YANK it!

[JohnOMDev]

after an hour of googling, looks like I found the right bug :) this is an issue when installing dbt-core==1.8.4, which lists Logbook==1.5.3 as a requirement.

Latest Logbook (1.7.0) seems to have fixed this but dbt-core still points to the older broken module.

Yeah. Exact same issue. dbt-core==1.8.3 also pointed to Logbook==1.5.3. I tried to install Logbook==1.7.0 seperately but had conflict issue. It is failing only my C1/CD and I believed dbt needed to fix it asap.

[miandreu]

Please YANK!!!!! We can implement five different workarounds for various use cases, but the solution shouldn´t be this one. YANK!!

[manugarri]

this is ridiculous, this package is one of the most core packages and it has over 12 years of runway. Im very surprised as this bug is obvious the moment a realisitic test build is perform.

Please yank this version.

[wyd3x]

Pls YANK it!! All my CI dead bcs of it

Crowdstrike round 2?

[delfick]

@chrisirhc we may wanna add a couple more dotpoints to the description of this issue

• the problem is with the package manager building wheels in an isolated environment and installing it's own version of setuptools for this purpose (which is why the version specified in your environment doesn't make a difference). The package manager ends up using the latest version of setuptools which doesn't have a feature used by some packages at build time. The PIP_CONSTRAINT environment variable mentioned below constrains what versions the package manager uses for this purpose
• Yanking this release is a popular request, the devs have yet to respond to this issue with what will happen moving forward. The solutions in this thread are only meant as temporary solutions until a real fix happens in setuptools.

edit: the version has been yanked https://github.com/pypa/setuptools/issues/4519#issuecomment-2255940870

[alpoi-x]

Maybe unpopular opinion - don't yank it! The test command has been deprecated for 5 years, which is more than enough time for people to update their packages. Clearly something drastic like this was necessary to force stragglers to make the change (else it wouldn't be an issue).

If the packages you are using don't resolve this in the near future, maybe consider not using unmaintained packages ¯_(ツ)_/¯

[Artui]

Maybe unpopular opinion - don't yank it! The test command has been deprecated for 5 years, which is more than enough time for people to update their packages. Clearly something drastic like this was necessary to force stragglers to make the change (else it wouldn't be an issue).

If the packages you are using don't resolve this in the near future, maybe consider not using unmaintained packages ¯(ツ)/¯

While I agree with this in generally, this issue explains the problem with this "deprecation" quite well? As in - it should've been at least "screaming" in the console each time the deprecated piece of code was utilised. That way consumers of packages with this deprecation could've reached out to maintainers, asking for upgrade or smth similar

This is a tricky one in that regard, however. Main problem is that a maintaner might've been using this command in a way that consumer wouldn't know about - and might've never encountered - and this deprecation notice, when the command runs, wasn't actually shown, because it's never called by consumers.

[712u3]

my poetry generates bad links for some reason, so fast and ugly workaround for today is

ENV PIP_CONSTRAINT=constraints.txt
RUN echo "setuptools<72" > constraints.txt && \
    poetry export --without-hashes -o requirements.txt && \
    sed -i 's/git+git+ssh/git+ssh/g' requirements.txt
RUN --mount=type=ssh pip install -r requirements.txt

seems that i should have also used --no-build-isolation but everything works well without it

p.s. don't yank) let's finally update things

[mwPandoraid]

p.s. don't yank) let's finally update things

holding prod environments hostage is not the best way to go about making package maintainers do their job is my opinion on the matter

[KPrasch]

I am having this issue with the autobahn package while installing with pip (https://github.com/crossbario/autobahn-python/issues/1644).

Changing my local projects requirements was not enough - the new version of setuptools was still being used despite pinning setuptools<72 for the reasons mentioned here https://github.com/pypa/setuptools/issues/4519#issuecomment-2255690881

Since I'm using poetry to build pip requirements.txt I was able to successfully generate and use constraints.txt as mentioned above (https://github.com/pypa/setuptools/issues/4519#issuecomment-2255652133).

Thanks for the help everyone! 😅

[sstegmueller]

FWIW the workaround option 1 exporting the PIP_CONSTRAINT=path/to/constraints.txt variable works as a workaround for pipenv too.

[stamparm]

not sure how to write this down without offending anyone. pretty much similar behavior as with python core devs - "[4 letter word] you, you've been warned in time, not our fault". some people just want to see the world burn

p.s. at the end we decided to use the dirty hack with PIP_CONSTRAINT in our production environment until the dust settles

[kalomaze]

[noiji]

for those using poetry, this helped solve the issue (thanks to many comments above)

echo "setuptools<72" > constraints.txt
export PIP_CONSTRAINT=constraints.txt
poetry run pip install ffmpy==0.3.2 #  package with the error
poetry install

but still dunno how to fix this in a cleaner way 🤨

[lauruskyte]

We are also using poetry, but did anyone solved it without creating a constraints.txt file?

[Afoucaul]

We are also using poetry, but did anyone solved it without creating a constraints.txt file?

I suppose you could do poetry run pip install "setuptools<72" then poetry run pip install --no-build-isolation <your package> - the constraints.txt file is only a static way of doing it

[manugarri]

@lauruskyte for the moment thats all there is. We are waiting to see if the breaking version gets janked.

[mgierada]

Oh this sucks so much!

[adamroyjones]

Not to be rude, but every comment in this thread (including this one, regrettably) leads to a large number of notifications and a large amount of noise that drowns out anything useful.

At this point, we should probably wait for a maintainer to provide an update.

[MarcinWieczorek]

This functionality has been deprecated for 5 years

[Jorden28]

For me the fix for the issue seems to be to run pip install --upgrade pip-tools pip wheel before running the command that runs into the ModuleNotFoundError: No module named 'setuptools.command.test' error message