ward fails to install with setuptools >= 58: error in cucumber-tag-expressions setup command: use_2to3 is invalid

[silverwind]

Installing this module is currently broken with python setuptools >= 58. It fails with EnvCommandError:

$ poetry add ward
...
error in cucumber-tag-expressions setup command: use_2to3 is invalid.
...

The dependency cucumber-tag-expressions relies on the removed use_2to3 option. It seems there is no fix available in cucumber-tag-expressions yet, so I guess that dependency would have to be dropped or forked to make it work again on setuptools >= 58.

[darrenburns]

Thanks for the report. This is now fixed in the latest release (0.64.0b0)

[AndydeCleyre]

I'm experiencing this in one of my projects, though relevant versions seem recent enough that it shouldn't happen.

Here's a containerized reproduction using podman:

$ podman run -it --rm --net=host docker.io/library/python:3:10 bash
# cd
# git clone https://github.com/AndydeCleyre/nestedtextto
# cd nestedtextto
# git checkout 6da027f6eacf299b3ad49f46444aa71d8f9c9393
# pip install -U pip pip-tools setuptools wheel
# python -V
Python 3.10.8
# pip list
Package    Version
---------- -------
build      0.9.0
click      8.1.3
packaging  22.0
pep517     0.13.0
pip        22.3.1
pip-tools  6.12.1
setuptools 65.6.3
tomli      2.0.1
wheel      0.38.4
# cd test

At this point, the following command succeeds:

# pip-compile --upgrade --no-header --annotation-style=line --strip-extras --allow-unsafe --resolver=backtracking test-requirements.in

arrow==1.2.3              # via -c ../nt2/requirements.txt, inform
click==8.1.3              # via click-completion, click-default-group, ward
click-completion==0.5.2   # via ward
click-default-group==1.2.2  # via ward
commonmark==0.9.1         # via -c ../nt2/requirements.txt, rich
cucumber-tag-expressions==4.1.0  # via ward
inform==1.27.0            # via -c ../nt2/requirements.txt, nestedtext
jinja2==3.1.2             # via click-completion
markupsafe==2.1.1         # via jinja2
nestedtext==3.5           # via -r test-requirements.in
pluggy==1.0.0             # via ward
plumbum==1.8.0            # via -r test-requirements.in
pprintpp==0.4.0           # via ward
pygments==2.13.0          # via -c ../nt2/requirements.txt, rich
python-dateutil==2.8.2    # via -c ../nt2/requirements.txt, arrow
rich==12.6.0              # via -c ../nt2/requirements.txt, ward
shellingham==1.5.0.post1  # via click-completion
six==1.16.0               # via -c ../nt2/requirements.txt, click-completion, inform, python-dateutil
tomli==2.0.1              # via -r ../nt2/toml-requirements.in, ward
tomli-w==1.0.0            # via -r ../nt2/toml-requirements.in
ward==0.67.0b0            # via -r test-requirements.in

But when I upgrade the nt2/requirements.txt lockfile, then try again:

# cd ../nt2
# pip-compile --upgrade --no-header --annotation-style=line --strip-extras --allow-unsafe --resolver=backtracking requirements.in
# git diff requirements.txt

diff --git a/nt2/requirements.txt b/nt2/requirements.txt
index 43ef311..9c6a786 100644
--- a/nt2/requirements.txt
+++ b/nt2/requirements.txt
@@ -5,10 +5,10 @@ commonmark==0.9.1         # via rich
 exceptiongroup==1.1.0     # via cattrs
 inform==1.27.0            # via nestedtext
 nestedtext==3.5           # via -r requirements.in
-plumbum==1.8.0            # via -r requirements.in
-pygments==2.13.0          # via rich
+plumbum==1.8.1            # via -r requirements.in
+pygments==2.14.0          # via rich
 python-dateutil==2.8.2    # via arrow, yamlpath
-rich==12.6.0              # via -r requirements.in
+rich==13.0.0              # via -r requirements.in
 ruamel-yaml==0.17.21      # via -r requirements.in, yamlpath
 ruamel-yaml-clib==0.2.7   # via ruamel-yaml
 six==1.16.0               # via inform, python-dateutil

# cd ../test
# pip-compile --upgrade --no-header --annotation-style=line --strip-extras --allow-unsafe --resolver=backtracking test-requirements.in

    error: subprocess-exited-with-error

    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [3 lines of output]
        /usr/local/lib/python3.10/site-packages/setuptools/dist.py:770: UserWarning: Usage of dash-separated 'upload-dir' will not be supported in future versions. Please use the underscore name 'upload_dir' instead
          warnings.warn(
        error in cucumber-tag-expressions setup command: use_2to3 is invalid.
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 64, in generate_metadata
    call_subprocess(
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
    raise error
pip._internal.exceptions.InstallationSubprocessError: python setup.py egg_info exited with 1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/piptools/scripts/compile.py", line 580, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/usr/local/lib/python3.10/site-packages/piptools/resolver.py", line 590, in resolve
    is_resolved = self._do_resolve(
  File "/usr/local/lib/python3.10/site-packages/piptools/resolver.py", line 622, in _do_resolve
    resolver.resolve(
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 373, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 204, in _get_updated_criteria
    self._add_to_criteria(criteria, requirement, parent=candidate)
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
    if not criterion.candidates:
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
    return bool(self._sequence)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
    candidate = func()
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
    super().__init__(
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 577, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 61, in prepare_distribution_metadata
    self.req.prepare_metadata()
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/req/req_install.py", line 541, in prepare_metadata
    self.metadata_directory = generate_metadata_legacy(
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 71, in generate_metadata
    raise MetadataGenerationFailed(package_details=details) from error
pip._internal.exceptions.MetadataGenerationFailed: metadata generation failed

[AndydeCleyre]

I'm guessing this is related to ward's requirement of rich: ^12.2.0, forcing the lockfile resolution to try an older version of ward (and so older version of cucumber-tag-expresssions) that is less restrictive regarding the rich requirement.

[AndydeCleyre]

So I think this is pretty much a repeat of #323