search

pypa / pip

The Python package installer
https://pip.pypa.io/
MIT License
9.47k stars 3.01k forks source link

pip fails to install source packages when the working directory contains a pyproject.toml file #11393

Open domdfcoding opened 2 years ago

domdfcoding commented 2 years ago

Description

When the current working directory contains a pyproject.toml file pip may fail to install (or build a wheel for) source packages. Setuptools tries to parse the pyproject.toml file from the working directory, but I'm not sure why. Because the configuration is for a different backend (and invalid for setuptools) setuptools raises an exception. The issue only seems to crop up when installing build dependencies to build wheels from sdists, but the build dependencies themselves can be wheels.

This worked correctly before pip 22.2. I have tracked the issue to 00e5adf2aa4223e37a191a454300d888af5687f5 (part of #11262).

I can narrow the issue down to the command (which pip invokes internally):

/tmp/setuptools-issue/bin/python3 /tmp/setuptools-issue/site-packages/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/foo --no-warn-script-location --no-binary :none: --only-binary :none: -i http://localhost:3141/root/staging -- 'setuptools >= 35.0.2'

/tmp/setuptools-issue is my virtualenv. The package being installed doesn't matter.

The crux of the issue seems to be that pip calls parse_config_files() in _distutils.py, which triggers setuptools to unnecessarily parse the pyproject.toml file.

I don't think this is an issue with setuptools, as it occurs regardless of the setuptools version, but if I uninstall setuptools or set SETUPTOOLS_USE_DISTUTILS=stdlib the issue goes away entirely, so there's some strange interaction between pip and setuptools' version of distutils going on.

Expected behavior

Pip correctly installs the package, or builds the wheel, as requested.

pip version

22.2.2

Python version

PyPy 3.7

OS

Ubuntu 20.04

How to Reproduce

  1. Create a pyproject.toml file with the following content:
[build-system]
requires = [ "whey",]
build-backend = "whey"

[project]
name = "demo"
version = "1.0.0"
dynamic = [ "requires-python", ]

[tool.whey]
python-versions = ["3.7", "3.8", ]
  1. run pip wheel msgpack==1.0.4

I can't reproduce on CPython, only on PyPy

Output

$ pip wheel msgpack==1.0.4 -v
Looking in indexes: http://localhost:3141/root/staging
Collecting msgpack==1.0.4
  Downloading http://localhost:3141/root/pypi/%2Bf/f5d/869c18f030202/msgpack-1.0.4.tar.gz (128 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.1/128.1 kB 131.2 MB/s eta 0:00:00
  Running command pip subprocess to install build dependencies
  Looking in indexes: http://localhost:3141/root/staging
  Collecting Cython~=0.29.30
    Downloading http://localhost:3141/root/pypi/%2Bf/eeb/475eb6f0ccf6c/Cython-0.29.32-py2.py3-none-any.whl (986 kB)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 986.3/986.3 kB 99.7 MB/s eta 0:00:00
  Collecting setuptools>=35.0.2
    Downloading http://localhost:3141/root/pypi/%2Bf/106/02cd0a6f5feab/setuptools-65.1.0-py3-none-any.whl (1.2 MB)
       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 114.6 MB/s eta 0:00:00
  Installing collected packages: setuptools, Cython
  ERROR: Exception:
  Traceback (most recent call last):
    File "/tmp/setuptools-issue/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
      status = run_func(*args)
    File "/tmp/setuptools-issue/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
      return func(self, options, args)
    File "/tmp/setuptools-issue/site-packages/pip/_internal/commands/install.py", line 470, in run
      pycompile=options.compile,
    File "/tmp/setuptools-issue/site-packages/pip/_internal/req/__init__.py", line 81, in install_given_reqs
      pycompile=pycompile,
    File "/tmp/setuptools-issue/site-packages/pip/_internal/req/req_install.py", line 758, in install
      prefix=prefix,
    File "/tmp/setuptools-issue/site-packages/pip/_internal/locations/__init__.py", line 256, in get_scheme
      prefix=prefix,
    File "/tmp/setuptools-issue/site-packages/pip/_internal/locations/_distutils.py", line 141, in get_scheme
      scheme = distutils_scheme(dist_name, user, home, root, isolated, prefix)
    File "/tmp/setuptools-issue/site-packages/pip/_internal/locations/_distutils.py", line 57, in distutils_scheme
      d.parse_config_files()
    File "/tmp/setuptools-issue/site-packages/_virtualenv.py", line 21, in parse_config_files
      result = old_parse_config_files(self, *args, **kwargs)
    File "/tmp/setuptools-issue/site-packages/setuptools/dist.py", line 868, in parse_config_files
      pyprojecttoml.apply_configuration(self, filename, ignore_option_errors)
    File "/tmp/setuptools-issue/site-packages/setuptools/config/pyprojecttoml.py", line 58, in apply_configuration
      config = read_configuration(filepath, True, ignore_option_errors, dist)
    File "/tmp/setuptools-issue/site-packages/setuptools/config/pyprojecttoml.py", line 136, in read_configuration
      return expand_configuration(asdict, root_dir, ignore_option_errors, dist)
    File "/tmp/setuptools-issue/site-packages/setuptools/config/pyprojecttoml.py", line 191, in expand_configuration
      return _ConfigExpander(config, root_dir, ignore_option_errors, dist).expand()
    File "/tmp/setuptools-issue/site-packages/setuptools/config/pyprojecttoml.py", line 238, in expand
      self._expand_all_dynamic(dist, package_dir)
    File "/tmp/setuptools-issue/site-packages/setuptools/config/pyprojecttoml.py", line 277, in _expand_all_dynamic
      for field in self.dynamic
    File "/tmp/setuptools-issue/site-packages/setuptools/config/pyprojecttoml.py", line 278, in <dictcomp>
      if field not in special
    File "/tmp/setuptools-issue/site-packages/setuptools/config/pyprojecttoml.py", line 322, in _obtain
      self._ensure_previously_set(dist, field)
    File "/tmp/setuptools-issue/site-packages/setuptools/config/pyprojecttoml.py", line 301, in _ensure_previously_set
      raise OptionError(msg)
  distutils.errors.DistutilsOptionError: No configuration found for dynamic 'requires-python'.
  Some dynamic fields need to be specified via `tool.setuptools.dynamic`
  others must be specified via the equivalent attribute in `setup.py`.
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 2
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /tmp/setuptools-issue/bin/python3 /tmp/setuptools-issue/site-packages/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-k8z8j5ng/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i http://localhost:3141/root/staging -- 'Cython~=0.29.30' 'setuptools >= 35.0.2'
  cwd: [inherit]
  Installing build dependencies ... error
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 2
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Code of Conduct

domdfcoding commented 1 year ago

I can still reproduce this with pip 22.3.1 and setuptools 65.6.3

uranusjr commented 1 year ago

Hmm this looks like a setuptools issue, but some investigation is needed to figure out a reproduction with bare setuptools. I have a suspicion this may be related to cwd.