Read modified conda dependencies from wheel metadata

[analog-cbarber]

Support some way for the wheel to publish information about its conda dependencies. Wheels configured in this way could be converted to conda packages with no other external information.

One way to do this would be using the Requires-External metadata key. The wheel METADATA file could contain lines like:

Rather than support the rename syntax in the metadata, we probably should just explicitly list names to add and names to drop. E.g.

Requires-External: conda-add; pytables>=3.10
Requires-External: conda-drop; tables

There don't appear to be any pyproject fields that map to this metadata, but there are some tools like auditwheel that can add these fields to existing wheels.

Note that PEP 725 proposes to add a optional-dependencies whose entries are PURL strings. There is a PURL format for conda,e.g:

pkg:conda/absl-py@0.4.1?build=py36h06a4308_0&channel=main&subdir=linux-64&type=tar.bz2

but since we are not actually describing an actual external dependency but an alternate dependency, we should probably add a type specifically for this package:

Requires-External: pkg:whl2conda/pytables@>=3.10
Requires-External: pkg:whl2conda/tables?drop=true

We could also provide a builder plugin to generate these entries from the pyproject file when the wheel is built or we could have an option of this program to add those entries to an existing wheel.

[analog-cbarber]

If a tool is generating the meta data entries, then it probably would be simplest for the tool to just generate all of the conda dependencies directly, but if they are being maintained using the PEP 725 mechanism, only added/dropped (or perhaps renamed) entries should be listed.

We can use pkg:whl2conda for the former case. And pkg:whl2conda-add, pkg:whl2conda-drop and pkg:whl2conda-rename for the latter. If any pkg:whl2conda entries are present then the regular wheel dependencies will be ignored along with any pkg:whl2conda-drop and pkg:whl2conda-rename.

[analog-cbarber]

It could be a long time before standard build tools support PEP 725 and poetry seems even less likely to bother to implement this, so it might be worth considering other mechanisms....

One possibility would be to use optional dependencies either using different keys:

[project.optional-dependencies]
whl2conda-add = [
   "pytables >=3.10"
]
whl2conda-drop = [
   "tables"
]

This would result in wheel METADATA:

Provides-Extra: whl2conda-add
Provides-Extra: whl2conda-drop
Requires-Dist: pytables >=3.10; extra == 'whl2conda-add'
Requires-Dist: tables; extra == 'whl2conda-drop'

Another option would be to just use an impossible version spec to indicate removal:

[project.optional-dependencies]
whl2conda = [
    "pytables >=3.10",
    "tables ===drop",
]

which would result in

Provides-Extra: whl2conda
Requires-Dist: pytables >=3.10; extra == 'whl2conda'
Requires-Dist: tables ===drop; extra == 'whl2conda'

Would be nice to also support some sort of rename declaration with wildcards, but I don't know that there is a good way to do that which would not violate the requirement that the dependencies be specified as valid PEP 508 strings.

[analog-cbarber]

Probably should really include some sort of support for renaming. Otherwise, users will have to list package dependency versions in two different places, which is highly likely to break.

One possibility would be to once again use the arbitrary equality version operator === with a special key:

[project.optional-dependencies]
whl2conda-rename = [
    "tables ==pytables" 
]

which would result in the following METADATA entries:

Provides-Extra: whl2conda-rename
Requires-Dist: tables ===pytables; extra == 'whl2conda-rename'

[analog-cbarber]

Looks like PEP621 support work is at least underway for poetry https://github.com/python-poetry/poetry/issues/9136

[analog-cbarber]

Note that poetry does not currently implement any syntax that maps to the "arbitrary equality clause" (===).

[analog-cbarber]

Another option would be to further abuse the extra namespace:

[project.optional-dependencies]
whl2conda-rename-tables = ["pytables"]

Provides-Extra: whl2conda-rename-tables
Requires-Dist: pytables; extra == 'whl2conda-rename-tables'