Some metrics of package quality

[ljharb]

Per your request:

• Of course, follow semver scrupulously. Known breaking changes must bump a major version. Any functionality or API additions must be in a minor bump. Everything else is a patch. Any breakages discovered must be rolled back and ported to the next major version. etc.
• npm install should be 100% of what's required in a given module to install all of its dependencies and/or dev dependencies
• npm test should always run tests
• npm explore foo && npm test should always run foo's tests (ie, always publish test files)
• Any build steps or compilation must be done prior to publishing - ie, coffeescript, babel, etc must never be required at runtime to use a module
• Any important project-related tasks should be discoverable from the scripts hash in package.json. Makefiles (or any ripoffs of same) are totally fine, but I shouldn't need to know how to use those to run 100% of important tasks: npm run foo can alias to make foo, for example.
• Good documentation: include a readme.
• package.json must always point to the git repo, to where issues can be filed, and must always include at least one license designation.

[iarna]

So caveats:

• If you're just looking at the scripts.test entry and not running it, remember that the default package.json made by npm init has one that's just set to fail automatically, so you'll have to check for that special case.
• If you actually run the npm explore foo form, you'd need something like – npm explore foo 'npm install && npm test' to get the dev deps.

[piranna]

Of course, follow semver scrupulously. Known breaking changes must bump a major version. Any functionality or API additions must be in a minor bump. Everything else is a patch. Any breakages discovered must be rolled back and ported to the next major version. etc.

On 0.x.y versions, the x should define backward incompatible API changes, the same way as major versions on post-1.0.0 releases.

[konobi]

All we can check is that the version is valid semver.

[piranna]

APIs could somewhat be check by analizing the methods and their parameters and see if they are added or they change, in that last case it should be in a new major version (or new 0.x). It could be also done to check deleted methods and params and show a warning if they were not marked as deprecated in a previous version.

[konobi]

If you can think of a way of doing that statically from an unbundled package tarball/repo, I'd certainly be interested.

[piranna]

Maybe using esprima or something similar... For prototype defined ones should be somewhat easy, for instance methods (this.foo = function(){}) would need more checking, but I think it's doable up to some degree.

[piranna]

Obviously being Javascript a dynamic language this could never be bullet-proof, but a statically analisis could be done in a high percentace of situations.

[konobi]

yeah, I already have some need for esprima to come along at some point... but this is probably one of those spots where a PR would work best (even if it's just tests).

[ljharb]

Look at the exported thing; compare Object.getOwnPropertyNames + Object.getOwnPropertySymbols and the typeof of their values (and name and length if they're functions, etc going into more depth if the types match), and then do the same walking up the prototype chain. That should allow you to produce a digest hash, as well as determine when observable properties are added, or deleted.

Added would guarantee at least a minor - deleted would guarantee a major - no observable changes is likely a patch.

Obviously for 0.x versions (which nobody should still be using anyways; dock them for that regardless), obviously the second number is for breaking changes, and the third for additions.