search

pypa / setuptools

Official project repository for the Setuptools build system
https://pypi.org/project/setuptools/
MIT License
2.5k stars 1.19k forks source link

[BUG] Cannot parse package marker expression in requirements.txt #4675

Open joseguerr opened 1 week ago

joseguerr commented 1 week ago

setuptools version

setuptools==75.1.0

Python version

Python 3.11

OS

macOS

Additional environment information

Description

After updating greenlet to 3.1.1 the marker expression is set to:

greenlet==3.1.1; python_version < '3.13' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))

It's not the best to build the expression but it's still valid. To which I get the error in the output section.

If I change to manually include start and ending parenthesis, i.e: greenlet==3.1.1; (python_version < '3.13' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32'))))))

I no longer have the issue.

Expected behavior

Expected setuptools to parse the marker expression

How to Reproduce

  1. install / Update greenlet to 3.1.1 and generate requirements.txt
  2. build a package that is using those same requirements.txt
  3. pip wheel --no-deps -w dist .

Output

  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [96 lines of output]
      /Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/setuptools/__init__.py:94: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
      !!

              ********************************************************************************
              Requirements should be satisfied by a PEP 517 installer.
              If you are using pip, you can try `pip install --use-pep517`.
              ********************************************************************************

      !!
        dist.fetch_build_eggs(dist.setup_requires)
      [pbr] Generating ChangeLog
      /Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/setuptools/command/develop.py:41: EasyInstallDeprecationWarning: easy_install command is deprecated.
      !!

              ********************************************************************************
              Please avoid running ``setup.py`` and ``easy_install``.
              Instead, use pypa/build, pypa/installer or other
              standards-based tools.

              See https://github.com/pypa/setuptools/issues/917 for details.
              ********************************************************************************

      !!
        easy_install.initialize_options(self)
      /Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
      !!

              ********************************************************************************
              Please avoid running ``setup.py`` directly.
              Instead, use pypa/build, pypa/installer or other
              standards-based tools.

              See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
              ********************************************************************************

      !!
        self.initialize_options()
      Marker evaluation failed, see the following error.  For more information see: http://docs.openstack.org/pbr/latest/user/using.html#environment-markers
      Error parsing
      Traceback (most recent call last):
        File "/Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/packaging/markers.py", line 199, in __init__
          self._markers = _normalize_extra_values(_parse_marker(marker))
                                                  ^^^^^^^^^^^^^^^^^^^^^
        File "/Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/packaging/_parser.py", line 255, in parse_marker
          return _parse_full_marker(Tokenizer(source, rules=DEFAULT_RULES))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/packaging/_parser.py", line 259, in _parse_full_marker
          retval = _parse_marker(tokenizer)
                   ^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/packaging/_parser.py", line 268, in _parse_marker
          expression = [_parse_marker_atom(tokenizer)]
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/packaging/_parser.py", line 284, in _parse_marker_atom
          with tokenizer.enclosing_tokens(
        File "/Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/contextlib.py", line 144, in __exit__
          next(self.gen)
        File "/Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/packaging/_tokenizer.py", line 187, in enclosing_tokens
          self.raise_syntax_error(
        File "/Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/packaging/_tokenizer.py", line 165, in raise_syntax_error
          raise ParserSyntaxError(
      packaging._tokenizer.ParserSyntaxError: Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after marker expression
          (python_version < '3.13' and platform_machine == 'aarch64' orplatform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "/Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pkg_resources/__init__.py", line 1652, in evaluate_marker
          marker = packaging.markers.Marker(text)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/packaging/markers.py", line 217, in __init__
          raise InvalidMarker(str(e)) from e
      packaging.markers.InvalidMarker: Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after marker expression
          (python_version < '3.13' and platform_machine == 'aarch64' orplatform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "/Users/joseguerra/hellofresh/uk-utils/uk-utils/.eggs/pbr-6.1.0-py3.11.egg/pbr/core.py", line 105, in pbr
          attrs = util.cfg_to_args(path, dist.script_args)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/joseguerra/hellofresh/uk-utils/uk-utils/.eggs/pbr-6.1.0-py3.11.egg/pbr/util.py", line 274, in cfg_to_args
          kwargs = setup_cfg_to_setup_kwargs(config, script_args)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/joseguerra/hellofresh/uk-utils/uk-utils/.eggs/pbr-6.1.0-py3.11.egg/pbr/util.py", line 464, in setup_cfg_to_setup_kwargs
          if pkg_resources.evaluate_marker('(%s)' % env_marker):
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/joseguerra/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pkg_resources/__init__.py", line 1655, in evaluate_marker
          raise SyntaxError(e) from e
      SyntaxError: Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after marker expression
          (python_version < '3.13' and platform_machine == 'aarch64' orplatform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
      error in setup command: Error parsing /Users/joseguerra/hellofresh/uk-utils/uk-utils/setup.cfg: SyntaxError: Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after marker expression
          (python_version < '3.13' and platform_machine == 'aarch64' orplatform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.
abravalheri commented 3 days ago

Hi @joseguerr could you please try to reduce the reproducer by removing any unnecessary 3rd party code? Please see https://stackoverflow.com/help/minimal-reproducible-example.

I suspect that the only thing you need to do is to ensure you have the latest version of packaging installed (or that packaging is not installed at all, so that setuptools uses its own vendored version). This is conclusion is reached via the following test:

cd /tmp
python3.12 -m venv .venv
.venv/bin/python -m pip install -U pip setuptools
.venv/bin/python -m pip uninstall packaging  # Let's make sure setuptools uses the bundled version of packaging
.venv/bin/python
Python 3.12.7 (main, Oct  1 2024, 08:52:11) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools # => changes sys.path so that vendored packaging is exposed
>>> from packaging.requirements import Requirement
>>> req = "greenlet==3.1.1; python_version < '3.13' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))"
>>> Requirement(req)
<Requirement('greenlet==3.1.1; python_version < "3.13" and platform_machine == "aarch64" or (platform_machine == "ppc64le" or (platform_machine == "x86_64" or (platform_machine == "amd64" or (platform_machine == "AMD64" or (platform_machine == "win32" or platform_machine == "WIN32")))))')>

Another part that is a bit worrying about your logs is the following:

  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.

Here we see that a deprecated feature is used, the deprecated feature in question is the CLI capabilities setup.py[^1]. This seems to indicated that a pyproject.toml file is missing. Please make sure your project has one. You can find more info in https://setuptools.pypa.io/en/latest/build_meta.html.

[^1]: setup.py is a valid configuration file for setuptools, but using it as a CLI tool is deprecated and as such is no longer fully supported.