what to do about `Requires-External` core metadata field?

[rgommers]

It turns out there is an existing user, setuptools-ext - see https://discuss.python.org/t/pep-725-specifying-external-dependencies-in-pyproject-toml/31888/18. From that discussion:

PEP 621 – Storing project metadata in pyproject.toml doesn’t currently have anything to say about supplying values for this field, but there are users: setuptools-ext is a build backend which exists mainly to specify this field in pyproject.toml using a tool subsection

Thank you for pointing that out. I am unsure, it’s the first time I hear about this field being used. I’m hoping someone with more experience with core metadata can weigh in about whether this field can and should be reused.

[rgommers]

Out of all comments on the PEP so far, this is the one I'm least sure about. On the one hand, the current use of Requires-External by setuptools-ext looks quite unusual, there are no real semantics as far as I can tell. E.g., in the README example, there are two packages (libpng and make) and one name of a programming language (C):

requires-external = [
    "C",
    "libpng (>=1.5)",
    'make; sys_platform != "win32"'
]

Given the lack of prescribed semantics, this only seems to be useful for a build backend and a metadata consumer that explicitly know about each other.

On the other hand, backwards compatibility matters, and it'd be much better to not break anything for setuptools-ext users (there don't appear to be many, 53 downloads last month says https://pypistats.org/packages/setuptools-ext).

@wimglenn how disruptive would it be for you and the users you know of if this field got repurposed?

@pradyunsg do you have any advice here? Are there precedents for this kind of thing?

[wimglenn]

The example with C, libpng, make is only due to reusing the existing "official" example of Requires-External usage, which has been there for as long as I can remember. setuptools-ext propagates those lines (and some other fields) from the pyproject.toml file into the package metadata and hence into the wheel/sdist, since vanilla setuptools can not.

I've never publicized setuptools-ext anywhere, and given the nil star count on GitHub I wouldn't be surprised if we're the only user. However, it is used internally at my employer fairly extensively (packaging automation for order of thousands packages). You wouldn't see any downloads of setuptools-ext in PyPI stats for this because we mirror and cache packages we need internally in our own index server.

this only seems to be useful for a build backend and a metadata consumer that explicitly know about each other

That's quite right, and it's exactly the case here. We also control the host OS/platform, so the external dependencies which may be specified are from a well-defined collection.

how disruptive would it be for you and the users you know of if this field got repurposed?

It's my expectation that as soon as setuptools implemented support for PEP 725, then setuptools-ext (as a setuptools wrapper) in it's current form would probably just break. If the field got repurposed I'd probably make a new release which cooperates with setuptools again and do something like

[build-system]
requires = ["setuptools-ext > 0.5"]
build-backend = "setuptools_ext"

Or, I'd just kill the project and find some other way to do what it was doing, passing around the relevant info without using core metadata fields at all. It would be mildly disruptive/annoying at $EMPLOYER, but as (probably) the only consumer of the field, not very disruptive in the grand scheme of things.

[rgommers]

Thank you @wimglenn, that is very helpful context, much appreciated.

Then it seems like it's at least feasible to reuse the field. We'll see if there are more comments about that choice.