Plan to support setup.py and remaining Python dependency lockfiles

google / deps.dev

Resources for the deps.dev API

https://deps.dev

Apache License 2.0

241 stars 18 forks source link

Plan to support setup.py and remaining Python dependency lockfiles #102

Open rtfpessoa opened 1 month ago

rtfpessoa commented 1 month ago

Currently the deps.dev UI and APIs have a big percentage of python versions maybe 20-30% which have no known dependencies. For Python specifically, this is mostly due to lack of support for setup.py, setup.cfg and some other dependency lockfiles.

Is there a plan to support these files?

rtfpessoa commented 2 weeks ago

@cuixq maybe you can help?

cuixq commented 2 weeks ago

@PFCM knows more about this :)

rtfpessoa commented 2 days ago

Maybe while I wait for the answer I can provide some details from my side. We are very interested in this support and are currently adding parsing capabilities to osv-scanner to:

Support for parsing the install_requires key in setup.py and setup.cfg
In setup.py it only supports arrays of plain string values being passed as a named argument
Any dependencies described in other requires keys are not scanned
Fails fast on unsupported inputs

What is the deps.dev source for deps? Does it download the binaries/releases and parses the source with osv-scanner? If yes, @cuixq do you think osv-scanner would accept a PR to add support for this?

cuixq commented 2 days ago

I think deps.dev fetches data from upstream (see https://docs.deps.dev/api/v3/#data) and computes dependency graph based on that.

Contribution is always welcome, so feel free to send the PR to OSV-Scanner!

rtfpessoa commented 2 days ago

@cuixq Any idea about the code that does this for python? I tried to look for it but could not find it.

The information, at least in good part only exists in the sdist binaries on pypi, not through APIs for this cases.

cuixq commented 2 days ago

I don't think the code for Python is open sourced.