protontypes / LibreSelery

Continuous distribution of funding to your project contributors and dependencies. Integrated into GitHub Actions
GNU Affero General Public License v3.0
124 stars 18 forks source link

Use the Github API to find the URL of dependencies #166

Open Ly0n opened 4 years ago

Ly0n commented 4 years ago

One of the most complex parts of LibreSelery is dependency scanning. The performance of this function is quite bad at the moment:

-- pypi: gitpython [==3.1.7]

FOUND global-data-manager: 'https://github.com/SpencerTuft/global-data-manager' [SpencerTuft] -- pypi: pybraries [==0.2.2] FOUND global-data-manager: 'https://github.com/SpencerTuft/global-data-manager' [SpencerTuft] -- pypi: urlextract [==1.0.0] FOUND global-data-manager: 'https://github.com/SpencerTuft/global-data-manager' [SpencerTuft]

This relates to following issue: https://github.com/protontypes/libreselery/issues/98

Even for larger packages, we get bad results for scanning LibreSelery itself. That is why I would suggest removing bibliothecary, Libaries.io, and the many regular expression and parsing we developed with a simple Github API call:

https://docs.github.com/en/graphql/reference/objects#dependencygraphdependency

Extract Upstream URL from Target Repository --> Get Dependencies of Dependency based on a depth parameter -> Gather all URL

Pros:

Cons:

@cornerman @kikass13 @yarikoptic @krux02 @fdietze Please give some feedback.

kikass13 commented 4 years ago

@Ly0n that's not exactly true, dependency SCANNING is fine. The dependency lookup (for authors, contributors inside that dependency) is flawed.

as far as im concerned, i don't care. Whatever works in the end is fine. I mentioned in the issue that the https://libraries.io/ api works perfectly. So I would rather keep using it directly instead of using pybraries (who is the real culprit in #98 )