[BUG] pyproject.toml validation breaks pdm-pep517

[mgorny]

setuptools version

61.3.0

Python version

Python 3.10.4

OS

Gentoo Linux

Additional environment information

Tested on top of pdm-pep517 git (pdm-project/pdm-pep517@04ed8589ad4400422ac9ca4c09b3ecb0df8edfab). pdm-pep517 bug report: https://github.com/pdm-project/pdm-pep517/issues/81

Description

pdm-pep517 happens to use setuptools internally to build projects with C extensions. However, these builds are now failing due to setuptools attempting to validate pyproject.toml according to its own rules, while it apparently uses an incompatible pdm-specific format. An example error is:

configuration error: `project` must not contain {'license-expression'} properties

Expected behavior

Not sure. Possibly skipping validation if pyproject.toml uses a different build system.

How to Reproduce

git clone https://github.com/pdm-project/pdm-pep517
cd pdm-pep517
python -m venv .venv
. .venv/bin/activate
pip install pdm
pdm run pytest tests

Output

The tests initially fail due to empty homepage:

-------------------------------------------------------- Captured stderr call ---------------------------------------------------------
/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py:102: _ExperimentalProjectMetadata: Support for project metadata in `pyproject.toml` is still experimental and may be removed (or change) in future releases.
  warnings.warn(msg, _ExperimentalProjectMetadata)
For maximum compatibility please make sure to include a `scheme` prefix in your URL (e.g. 'http://'). Given value: 
configuration error: `project.urls.homepage` must be url
Traceback (most recent call last):
  File "/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/setup.py", line 29, in <module>
    setup(**setup_kwargs)
  File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 122, in setup
    dist.parse_config_files()
  File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 854, in parse_config_files
    pyprojecttoml.apply_configuration(self, filename, ignore_option_errors)
  File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 54, in apply_configuration
    config = read_configuration(filepath, True, ignore_option_errors, dist)
  File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 120, in read_configuration
    validate(subset, filepath)
  File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 43, in validate
    raise error from None
ValueError: invalid pyproject.toml config: `project.urls.homepage`

After fixing it not to be empty, they fail due to pdm-specific keys, e.g.:

/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py:102: _ExperimentalProjectMetadata: Support for project metadata in `pyproject.toml` is still experimental and may be removed (or change) in future releases.
  warnings.warn(msg, _ExperimentalProjectMetadata)
configuration error: `project` must not contain {'license-expression'} properties
Traceback (most recent call last):
  File "/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/setup.py", line 29, in <module>
    setup(**setup_kwargs)
  File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 122, in setup
    dist.parse_config_files()
  File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 854, in parse_config_files
    pyprojecttoml.apply_configuration(self, filename, ignore_option_errors)
  File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 54, in apply_configuration
    config = read_configuration(filepath, True, ignore_option_errors, dist)
  File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 120, in read_configuration
    validate(subset, filepath)
  File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 43, in validate
    raise error from None
ValueError: invalid pyproject.toml config: `project`

[abravalheri]

Thanks for reporting @mgorny.

It seems that @frostming was faster than me and implemented a workaround in pdm-pep517 😄 (thank you very much Frost!)

@frostming, if you prefer a different approach, I don't mind disabling the validation in setuptools, but we have to agree on a mechanism for the wrapper to communicate with setuptools[^1].

We also need to ensure that some validation is done somewhere (there are code paths in the setuptools implementation that rely on the existence of the validation).

[^1]: Checking the value of [build-system] build-backend does not sound like a good solution for me: some lightweight wrappers might want the validation to happen while others want to completely take over...

[mgorny]

To be honest, I don't think the workaround fully fixes the problem — it silences the test failure but some real package could still fail if it used C extensions in combination with incompatible pyproject.toml keys.

[abravalheri]

I am more than happy to tackle this at setuptools level. I just would like us to agree on the approach.

PEP 517 acknowledges the existence of backend wrappers, which means that it is perfectly fine for the value of [build-system] build-backend to be other than setuptools.build_meta and setuptools still be the main "driver" behind the build[^1].

With that in mind:

(a) Lightweight wrappers should rely on the same validations that setuptools uses by default. (b) Other wrappers might want to take over and disable the validations or the whole processing of pyproject.toml.

I would like to agree on a approach to unequivocally identify the situation in (b), then we can make setuptools step aside.

[^1]: This is true not only for small in-tree wrappers, but also for other backends that extend setuptools via entry-points, e.g. pbr.