search

jfrog / gocenter

The Github README for JFrog Go-center. Use this for reporting issues
https://gocenter.io
Apache License 2.0
164 stars 23 forks source link

Proposal: Add scores to Go Modules in GoCenter #14

Open retgits opened 5 years ago

retgits commented 5 years ago

Proposal: Add scores to Go Modules in GoCenter

Last update: 2019-06-22

Introduction

Starting with two quotes from Russ Cox as he explained his thoughts on software dependencies:

For decades, discussion of software reuse was far more common than actual software reuse. Today, the situation is reversed: developers reuse software written by others every day, in the form of software dependencies, and the situation goes mostly unexamined.

(...)

Software dependencies carry with them serious risks that are too often overlooked.

The difficulty is figuring out which criteria are important to modules and how well those criteria are taken care of in a specific Go module. I propose to add a module score to GoCenter based on the criteria the community deems important calculated in using a formula that is open to discussion and updates as both Go evolves and requirements on modules evolve.

Benefits to the Go community

Adding a score based on predetermined criteria will benefit developers in seeing how well a module is perceived and enables authors of modules to identify possible areas of improvement.

Determining the score

The score will be calculated by GoCenter once per week, with the ability on the UI to trigger an update. The criteria that will make up the score are:

Criterium Why Scale
Number of dependents A dependency which is highly popular is usually a good indicator for having good quality 5 dependent projects equals 1 point
Average score of direct dependencies High quality nodules will use high quality dependencies themselves Average score of dependencies
Number of GitHub stars While GitHub stars are considered vanity metrics, they do give some indication into the popularity of the module 100 stars equals 1 point
Number of downloads Modules that get downloaded frequently and are thus trusted and used by others, means the module is high quality 100 downloads equals 1 point
Deleted versions While modules will remain in GoCenter, not having the version available in the version control system is an indication of untrustworthy modules 0 points if versions exist in GoCenter but not in the version control system, otherwise 5 points
Deleted module While modules will remain in GoCenter, not having the project available in the version control system is an indication of untrustworthy modules 5 points if the project exists, total score will automatically be 0 if the project is deleted
Documentation Knowing how to use a module is important 5 points when a README exists, 0 points if none exists
Maintainers Having active maintainers for a module is important to make sure updates can be done 5 points for a number of maintainers higher than 1, 0 points otherwise
Linting score Properly formatted and linted code is important for the readability and thus quality of the code Similar to Go ReportCard points will be awarded depending on the scored percentage. 5 points -> score of 97% or higher, 4 points -> score between 81% and 96%, 3 points -> score between 61% and 80%, 2 points -> score between 50% and 60%, 1 point -> score between 31% and 50%, 0 points otherwise
Activity Projects with a high number of commits, and active development are likely to be better quality modules 5 points when commits are made in the last two weeks, one point is deducted for every two weeks without a commit
Releases Projects that actively release new versions are likely to be better quality 5 points when releases are made in the last two months, one point is deducted for every two months without a release
Tests Successful and stable projects have tests and have a high test coverage 5 points are awarded to a test coverage score of over 80%, one point is deducted for every 10% less than that

The score of the module is determined by grading it against a normal distribution of all scores modules that GoCenter knows about.

Percentile range Score
96th - 100th percentile A+
86th - 95th percentile A
61sh - 85th percentile B
31st - 60th percentile C
0th - 30th percentile D

Community feedback

At JFrog's user conference, we've had a board with some of these criteria listed (see image below) and the feedback we got was:

score

klasyap commented 5 years ago

Based on the Go community's feedback, we are going to incorporate module scores for all Go modules in GoCenter. You can view the proposal here. We invite the Go community to comment and/or submit a pull request against the scoring formula.