search

gojp / goreportcard

A report card for your Go application
https://goreportcard.com
Apache License 2.0
2k stars 250 forks source link

Listing popular Go repositories dynamically. #211

Closed tariq1890 closed 3 years ago

tariq1890 commented 7 years ago

Currently, the popular report cards on the go report card homepage are hard-coded. Is it possible to retrieve this dynamically instead? We could make a GitHub API call to retrieve popular/trending repositories. Once retrieved we can cache the list of repositories in boltdb and refresh it every week or so.

If you're okay with this idea, I can work on a PR for this.

hermanschaaf commented 7 years ago

Thanks for the suggestion @tariq1890, I like the idea as you described it, where we can pull the most popular Go repositories from Github every week or so. I'd be open to trying it if the new code can be added as a separate command-line tool that updates boltdb. Of course there will also be some minor changes in the existing code to then read from the database, or fall back to defaults if none are found.

One thing that we may have to be careful of with this plan is that not all Go repositories have the main code at the root of the Github URL, or in other words, the full go get path may be different from the repo URL. An example would be the Prometheus Go client, https://github.com/prometheus/client_golang . To get around this, the command-line tool might need to also check that the report can be successfully generated. This would have the additional benefit that front-page reports have been cached, which is good for performance.

If you'd like to make a PR for this tool, I'd be for it :+1:

tariq1890 commented 7 years ago

@hermanschaaf That's a great idea. Testing if a report is generatable and populating the cache with it. I can do the PR for that. Thanks for the tip!

tariq1890 commented 7 years ago

@hermanschaaf How would you like to gather the popular/trending repos? Unfortunately, there is no way to retrieve trending repos through the GitHub API. Other open source repositories seem to be scraping the github trending page HTML. The only drawback is that when GitHub changes their HTML structure, the scraping logic may cease to work and necessitate an updating of code logic.

If you're okay with the scraping approach, I can go ahead and get around to implementing. I personally feel that the scraping approach is fine since we are implementing this a command line tool. So any changes in the github.com HTML will not break the goreportcard. Your thoughts?

hermanschaaf commented 7 years ago

I agree, since there are no other options and it's a command-line tool, HTML scraping is okay in this case. Sounds good!

tariq1890 commented 7 years ago

Great! I've made this preliminary PR so that I can go ahead work on the popular repos command line tool. Let me know if you have any feedback. Thanks!

https://github.com/gojp/goreportcard/pull/213

github-actions[bot] commented 3 years ago

This issue is stale because it has been open 60 days with no activity.