search

python-wheel-build / fromager

Build your own wheels
https://fromager.readthedocs.io/en/latest/
Apache License 2.0
7 stars 11 forks source link

packagesettings: pypi sources and package settings download_source overrides should allow for validating hashes #475

Open pnasrat opened 1 month ago

pnasrat commented 1 month ago

I had to use the download_source override for a package build recently and it got me thinking if hash support should be added

I was thinking of something like the following 

download_source:
    url: https://egg.test/${canonicalized_name}/v${version}.tar.gz
    destination_filename: ${canonicalized_name}-${version}.tar.gz
    hashes:
      "1.0.1":
        "sha256": e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
      "1.0.2":
        "sha256": e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

That would get added to packagesettings possibly 

# Package source hashes
Hashes = dict[str, str]
VersionHashes = Mapping[PackageVersion, Hashes]

However when I started to look at the codebase to add it doesn't look like _download_source_check validates the hashes of downloads from pypi, so before proceeding further I wanted to open for discussion.

dhellmann commented 1 month ago

Manually managing those would eliminate some of the benefits of the templated URL feature, since you would have to edit the config anyway. OTOH, I could see some value in being able to check the signatures.

How does a user determine the hash values? Could fromager get them that way, too?

pnasrat commented 1 month ago

Ignoring the override case adding support for sources that are on PEP 503 compliant indexes then the hash will be in the urlfragment. or per PEP 714 uses core-metadata for hashes.

Currently get_project_from_pypi in resolver.py would have the access to the href and attributed to do that, but wiring that through so the source download code has access to i

For other url overrides that would really depend on the external source - in my case I noticed this building dask-gateway which needed both the source download and build_dir set as the source download on pypi doesn't include theh requirements.txt that its setup.py loads. I set it to the github tag archive download url.

I understand your concern about manually managing these, and adding them would only make sense if archive validation was being done on the current source downloads.

pnasrat commented 1 month ago

It looks like some work on extra metadata into candidates is in

https://github.com/python-wheel-build/fromager/pull/464/