[BUG] Many packages are no longer installable after test command is removed

[chrisirhc]

For those landing on this issue, please see: (thank you @delfick for summarizing this)

• This functionality has been deprecated for 5 years, there is a separate issue for discussing if there would have been a better way of removing the functionality [FR] More gradual breakage of setuptools.command.test #4520
• For people using pip (or tools using pip, like poetry), PIP_CONSTRAINT set to a file with setuptools<72.0 should work
• For people using uv, there seems to be a bug where UV_CONSTRAINT doesn't affect the version of setuptools used in build isolation (see ModuleNotFoundError: No module named 'setuptools.command.test' astral-sh/uv#5551) and in that case the solution is either --no-build-isolation (which should be considered a temporary solution) or getting packages to not use setuptools.command.test or using forks of those packages that comment out the use of setuptools.command.test (until setuptools releases a fix/revert)
• To press subscribe to the ticket and/or thumbs up the top post instead of adding "same" comments

Quoted from https://github.com/pypa/setuptools/issues/4519#issuecomment-2255389347

---

setuptools version

setuptools==72.0.0

Python version

Python 3.9

OS

Linux

Additional environment information

No response

Description

The breakage change released on 72.0 breaks the default build isolation build of many packages since many of these packages do not pin on a particular setuptools version. There is also no way to pin to an older setuptools version as pip doesn't offer a way to do this. Installing setuptools==71 first, then installing the package doesn't work as the default build isolation resolves the dependencies without considering the lock file nor currently installed packages.

Based on these packages need to be patched with setup_requires before install, which is nearly impossible since we'd need to patch all packages.

Expected behavior

Install is successful.

How to Reproduce

1. Create a virtual environment
2. Upgrade to latest pip 24.2: python -m pip install --upgrade pip
3. pip3.9 install doubles==1.4.0

Output

me@my ~/repro-pip-fail
 % pip3.9 install doubles==1.4.0                    
Looking in indexes: http://…
Collecting doubles==1.4.0
  Using cached http://…/doubles-1.4.0.tar.gz (16 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/user/repro-pip-fail/venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 497, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-h9ppcze_/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 2, in <module>
      ModuleNotFoundError: No module named 'setuptools.command.test'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[SarahAndersonValerann]

For anyone using poetry install in their docker images and are blocked by this, we were able to resolve this by creating a python package dependent on setuptools<72.0.0 and adding that as a dependency to the poetry that doesn't build. simply adding setuptools<72.0.0 to the blocked poetry didn't work, but seems to if it's the dependent of another package

[keyz182]

This functionality has been deprecated for 5 years

Unfortunately this is what leaving something in a long-term state of deprecation leads to. It becomes de-prioritized.

If I have code that's been deprecated for an extended period, I prefer to change the deprecation notice to a stronger one including a date when we're ready to remove it, which prompts people to re-prioritize.

No need to be laying blame anywhere here though, rather look to recommendations to make the process smoother in future.

In terms of workarounds, PIP_CONSTRAINTS alone didn't work for us with poetry, We had to do the following in our pipelines:

    - echo "Init Poetry and create requirements"
    - poetry config virtualenvs.in-project true
    - poetry config <snip>
    - poetry export --with dev -o requirements.txt --without-hashes
    - poetry export --with dev -f constraints.txt -o constraints.txt --without-hashes
    - PIP_CONSTRAINT="$(pwd)/constraints.txt" poetry run pip install -r requirements.txt

[manugarri]

simply adding setuptools<72.0.0 to the blocked poetry didn't work, but seems to if it's the dependent of another package

Interesting, so if i understood, on a separate package you pinned setuptools to <72.0.0 , and when installing in an environment that has that other package as a dependency, it respects the setuptools constraint?

[agronholm]

For me the fix for the issue seems to be to run pip install --upgrade pip-tools pip wheel before running the command that runs into the ModuleNotFoundError: No module named 'setuptools.command.test' error message

Why are you installing wheel?

[jaraco]

Thanks for the report. I've yanked the 72.0.0 release while I investigate and determine if there's a less invasive way to remove the command.

[jaraco]

Next, let me acknowledge - I did expect this change to have some disruption, but not nearly as much as it had. Sorry for the disruption.

I was anticipating that impacted users would have had visibility to the deprecation warnings and most packages would long ago have migrated and that the handful that remained could be addressed promptly. It seems instead the deprecation warning wasn't placed with enough visibility for those installing affected packages, so the message only served to reduce reliance on the command for running tests, but not necessarily removing the command entirely. This statement is not an excuse, just context.

[Pragith]

For me the fix for the issue seems to be to run pip install --upgrade pip-tools pip wheel before running the command that runs into the ModuleNotFoundError: No module named 'setuptools.command.test' error message

Thanks @Jorden28, this worked for me.

[matt-phylum]

This function can never be removed because removing it renders old package versions non-installable. It's already too late to warn past developers about deprecation.

Python packaging has a huge problem with semantic versioning. The official documentation, to this day), recommends to always depend on the latest version of setuptools regardless of major version (and setuptools is not the only build backend doing this). This means that changes to setuptools can unexpectedly break nearly any sdist package, in this case rendering them non-installable. The change to setuptools version 72.0.0 indicates that it is not compatible with old packages, but the old packages accept any version and don't even specify a single version that is known to be compatible. The builder has no better option than to install 72.0.0 and hope.

The simple way to fix this would be to create a setuptools2 and keep setuptools as a compatible version for packages that use it. However, even if the documentation says to specify a compatible major version, developers may continue not doing it because it's convenient to them at the time, leading to this problem again years later. Maybe setuptools2 can error out if it detects that it is being required without a major version specified to avoid having to create a setuptools3.

To keep the name setuptools, maybe setuptools could detect that it is being required without a major version and replace itself with an older version or compatibility wrapper. It's probably good enough to require the version be set in pyproject.toml and assume anything without pyproject.toml build-backend should use the oldest compatibility mode.

It'd also be possible for the package builder to do this kind of thing if it detects an unconstrained build backend version, but every package builder would need to know about the compatible fallback versions of every package build backend and that sounds like a good way to create inconsistent behavior across builders.

[keyz182]

Next, let me acknowledge - I did expect this change to have some disruption, but not nearly as much as it had. Sorry for the disruption.

I was anticipating that impacted users would have had visibility to the deprecation warnings and most packages would long ago have migrated and that the handful that remained could be addressed promptly. It seems instead the deprecation warning wasn't placed with enough visibility for those installing affected packages, so the message only served to reduce reliance on the command for running tests, but not necessarily removing the command entirely. This statement is not an excuse, just context.

The good news is I think this has exposed what is maybe not a bug, but an oversight at least on my part - that despite having versions pinned throughout our code base and pipelines, isolated build of packages were not subject to those constraints.

[faizzed]

Coming here from https://github.com/AUTOMATIC1111/stable-diffusion-webui

Putting 👇 in requirements.txt fixed it for me.

setuptools==69.5.1

[alonme]

@chrisirhc please update issue to link to https://github.com/pypa/setuptools/issues/4519#issuecomment-2255940870 and https://github.com/pypa/setuptools/issues/4519#issuecomment-2255961257

[htelsiz]

epic morning

[berkaycamur]

I just wanted to deploy at Monday instead of Friday and thats what happened

[ParmarMiral]

The official version of setuptools is now 71.1.0 again. https://pypi.org/project/setuptools/#history :sweat_smile:

[asteel-gsa]

The official version of setuptools is now 71.1.0 again. https://pypi.org/project/setuptools/#history 😅

Confirmed local build and github workflow build is working again for our setup after this

[astrojuanlu]

This function can never be removed because removing it renders old package versions non-installable.

That is not correct, workarounds existed before setuptools 72.0.0 got yanked, see https://github.com/pypa/setuptools/issues/4519#issuecomment-2254983472

[LewisCowlesMotive]

I just found out now this has been yanked, so I don't know if my fix worked, or I thought I gained understanding, when there was more confusion.

I Think pinning outside of my pyproject.toml (with poetry) fixed because setuptools on the machine is used, rather than the one specified in poetry. Am I wrong about this?

Our original error had python 3.11 and 3.10 in the same traceback, so I poetry env use 3.11; no more nonsense about 3.10; but still not fixed.

I pinned in pyproject.toml, and nothing. I was hoping the transitive dependency pyvcr would as it's wildcarded, just use whatever is available.

I Also submitted them a PR; but they are being used via pytest-recording

I'm uncertain if this is that this installs package dependencies, so following category theory has to exist outside of that; or if the rollback fixed things.

[CaselIT]

Next, let me acknowledge - I did expect this change to have some disruption, but not nearly as much as it had. Sorry for the disruption.

I was anticipating that impacted users would have had visibility to the deprecation warnings and most packages would long ago have migrated and that the handful that remained could be addressed promptly. It seems instead the deprecation warning wasn't placed with enough visibility for those installing affected packages, so the message only served to reduce reliance on the command for running tests, but not necessarily removing the command entirely. This statement is not an excuse, just context.

Maybe having a daily pipeline that tries to install the top N packages (say top 250 or other reasonable value) in pypi using the develop branch would help catching widespread regressions like these?

[manugarri]

This function can never be removed because removing it renders old package versions non-installable.

That is not correct, workarounds existed before setuptools 72.0.0 got yanked, see #4519 (comment)

I disagree Juanlu (hola!) , many build systems are centralised , meaning customers (teams) can provide a pyproject , requirements.txt or similar, and the system takes care of the rest. This means individual python teams do not have the option to specify commands, setup environment variables or other custom changes. For example, in my current company, adding this hotfix would have involved adding quite some hacky code into a non python codebase, just to support a temporary fix.

[jaraco]

Maybe having a daily pipeline that tries to install the top N packages (say top 250 or other reasonable value) in pypi using the develop branch would help catching widespread regressions like these?

See https://github.com/pypa/setuptools/issues/4520#issuecomment-2256130396. There are some tests, but they're not as deep as you might hope. Probably that test could be expanded to the top 100 or top 250.

[jaraco]

I'm releasing the hot fix, which restores a small amount of the previous code to limit the breakage to "running setup.py test" but doesn't break building a package that is importing the command, adding a deprecation warning for that case. It's releasing as v72.1.0. I'll be keeping a close eye on this issue and the issue tracker in case that change doesn't adequately limit the disruption.

[davidaurelio]

@jaraco Does that mean that building old projects relying on this will break again? If that were the case, we'd end up with a broken CI pipeline once more. Since we rely on uv we have no means of controlling which version of setuptools will be pulled in when building projects.

In our case, one of the breaking packages was ffmpy: https://github.com/Ch00k/ffmpy/blob/master/setup.py#L2

Please hold off with a new release at least until uv provides a workaround.

[agronholm]

Does that mean that building old projects relying on this will break again? If that were the case, we'd end up with a broken CI pipeline once more. Since we rely on uv we have no means of controlling which version of setuptools will be pulled in when building projects.

In our case, one of the breaking packages was ffmpy: https://github.com/Ch00k/ffmpy/blob/master/setup.py#L2

Please hold off with a new release at least until uv provides a workaround.

No, IIRC only explicitly running python setup.py test would give an error, but merely having a setup.py referencing the test command would not.

[jaraco]

Given the lack of noise or reports since the release of 72.1.0, I believe the fix addressed the vast majority of use-cases affected by 72.0.0. My hope is that any remaining issues are isolated and readily addressable, but feel free to follow up here or in a new issue.

I want to thank everybody who was impacted by this for remaining civil, for your efforts in finding and documenting workarounds, and for providing constructive feedback on a change that was unnecessarily hostile to the ecosystem.

[nykcrystal]

In my case , I deupgrade my setuptools to version 71.1 solved the problem as well

[ziga7631]

In my case, the following 2 options work using Poetry:

1. First installing the problematic module and then the remaining ones:

   poetry run pip install logbook==1.5.3 --no-build-isolation
   poetry install

2. Specifying build-system requirement for Setuptools inside pyproject.toml:

   [build-system]
   requires = ["poetry-core", "setuptools<72.0.0"]
   build-backend = "poetry.core.masonry.api"

Should the second option also be working, or is this only some coincidence. It is by far the easiest solution using Poetry.

[justinclift]

@ziga7631 The problem release (72.0.0) was yanked yesterday. So in theory things should be ok now regardless. Just make sure your system isn't using 72.0.0 specifically.

There's also a newer 72.1.0 release with a fix in it as well.