Validating package name->cid mapping integrity

[andrew]

A bit of a follow on from my musings in https://github.com/protocol/package-managers/issues/5, mostly for myself to get things straight in my head but may be useful for others

With registry.js.ipfs.io users trust that the cid returned from https://registry.js.ipfs.io/bignumber is from the actual package that was originally uploaded to registry.npmjs.org, the npm client will check the downloaded package against the shasum field, but the users also trust that the shasum hash hasn't be messed with too.

Essentially they are trusting Protocol Labs that isn't tampering with the registry data that it's replicating, which includes ensuring that the code and servers running registry.js.ipfs.io is well secured to stop intruders getting in and adding/replacing malicious metadata.

This is all fine if Protocol Labs wants to be a trusted source of package metadata for multiple package managers, but I suspect that's not the goal, instead getting to a point where the mirroring of package contents and metadata is distributed.

But distributing metadata is tricky because if someone adds a mapping of: foobar-1.1.0.tgz => 'QmZKxosx..., how can you trust that they didn't mess with the original, two different ways that I can see:

• Don't trust any existing name->cid mappings on ipfs: always calculate cids yourself from packages that you've downloaded from the upstream registry, then store those cids somewhere for later resolution (manifest files in a code base or within a database in your own mirror/proxy). I believe this is basically what gx does, although Go doesn't have a traditional registry with immutable releases to mirror from
• Trust but verify: users can use any name->cid mappings found on IPFS but they'd need to check the integrity of the package contents against the upstream registry to be sure it's correct, which means making http requests to the upstream registry to confirm.

For package managers that do support signing, confirming signatures might be one way to bypass this, or if the registry is signing packages that get uploaded then those signatures can be verified, for example: https://blog.npmjs.org/post/172999548390/new-pgp-machinery

[andrew]

Great quote from @olizilla that I wanted to note down:

the dream would be having the package manager lean on IPFS internally for integrity checking

[achingbrain]

Interesting that npm are signing packages using their own PGP key rather than that of the package author.

[andrew]

Looks like the Go team (Google) will start signing go packages as they are published at some point this year: https://blog.golang.org/modules2019

For publicly-available modules, we intend to run a service we call a notary that follows the module index log, downloads new modules, and cryptographically signs statements of the form “module M at version V has file tree hash H.” The notary service will publish all these notarized hashes in a queryable, Certificate Transparency-style tamper-proof log, so that anyone can verify that the notary is behaving correctly. This log will serve as a public, global go.sum file that go get can use to authenticate modules when adding or updating dependencies.

[warpfork]

Particularly interesting in the new Go work is the "queryable, Certificate Transparency-style tamper-proof log" part. Play scrabble with that and you can almost spell "merkle tree" there.