search

fedora-infra / fedora-packages

A webapp that allows searching packages in Fedora. Written in Python using TurboGears2 and Moksha.
https://apps.fedoraproject.org/packages
GNU Affero General Public License v3.0
58 stars 59 forks source link

Lookup Fedora package given PyPI package name from requirements.txt #420

Open abitrolly opened 5 years ago

abitrolly commented 5 years ago

I want to know if some PyPI package can be ported to Fedora packages. For that I need to analyze if dependencies are already packed. And for that I need to query all those dependencies as listed in requirements.txt and get Fedora package names for them. Is that possible?

pypingou commented 5 years ago

most python libraries will be packaged as python-<name>, so you starting with this model should get you pretty far.

abitrolly commented 5 years ago

Got pretty far already. Not sure if scripts are worthy packaging, but here is it.

https://github.com/repology/repology/issues/893#issuecomment-523852436

abitrolly commented 5 years ago

Got a pointer to standardized name https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_automatic_provides_with_a_standardized_name from https://discussion.fedoraproject.org/t/check-requirements-txt-against-existing-packages/2173/3

abitrolly commented 5 years ago

@pypingou how to expose Requires: field from package .rpm through API in fedora-packages? Or which component of fedora-infra can do this?

cverna commented 5 years ago

@pypingou how to expose Requires: field from package .rpm through API in fedora-packages? Or which component of fedora-infra can do this?

you should probably use mdapi for that https://mdapi.fedoraproject.org/

abitrolly commented 5 years ago

I could trace needed Requirements header down to /var/cache/dnf/fedora-8c9e363c32277c01/repodata/912f062d93e096c75901055ffca02a0c3961b33b8e1dd65319d97d493d3e49d5-primary.xml.gz where it is stored as XML entry.

<rpm:entry name="python3dist(straight.plugin)" flags="EQ" epoch="0" ver="1.4.0.post1"/>

I am not sure if Repology uses the same cache to extract information about Fedora packages. https://github.com/repology/repology/blob/613734c89c493bc5612d0e362c0019ad7f10984c/repology/parsers/parsers/repodata.py

Right now I am looking at mdapi to see if it allows lookups by python3dist.

abitrolly commented 5 years ago

It works! :D https://mdapi.fedoraproject.org/f30/provides/python3dist(straight.plugin)

abitrolly commented 5 years ago

@cverna why this data is not exposed in source repo metadata? https://github.com/repology/repology-rules/issues/253#issuecomment-524973909

cverna commented 5 years ago

@abitrolly I don't know :smile: maybe you should ask that to the releng folks (https://pagure.io/releng) or maybe @pypingou knows

pypingou commented 5 years ago

Because that info is introduced when the rpm is built and the source metadata contain the metadata about the source-rpm which is basically a tarball containing the spec file, the patches (if there are any) and the sources).

abitrolly commented 5 years ago

@pypingou is it possible to move python3dist into source metadata? The source belongs to upstream PyPI anyway.

pypingou commented 5 years ago

mdapi exposes what is in the repository metadata. Adding it to the soure repository (ie to the metadata of the source-rpm) is out of the scope of mdapi itself and I suspect not something that the RPM folks would be able to support.

abitrolly commented 5 years ago

Probably not for mdapi. Exposing PyPI source info in source metadata would help projects like https://repology.org maintain links between packages without hacks https://github.com/repology/repology-rules/commit/2c338bb0a8575e5a31b84581953b1e7116f6e29e