search

JamieMason / shrinkpack

Fast, resilient, reproducible builds with npm install.
https://www.npmjs.com/package/shrinkpack
MIT License
793 stars 38 forks source link

Verify shrinkpack-ed packages are the same as remote #51

Closed 5id closed 8 years ago

5id commented 8 years ago

Interested on your thoughts about having the ability to run something like shrinkpack --verify, which would attempt to redownload the original package tgz found in the npm-shrinkwrap.json file (rather than using the local), and compare the checksums against the files in node_shrinkpack.

This would give us confidence that what is stored in the repository, matches the packages remotely. My concern arises from when people are raising PR's to check in the node_shrinkpack files, there's no way to tell if they're safe or not without manually checking the checksums yourself.

Ideally once npm-shrinkwrap.json supports checksums this could all go away, and shrinkpack would continue working fine.

JamieMason commented 8 years ago

I'd be hesitant to have the scope of shrinkpack creep too far I think @5id. I think there could be a case though for someone taking these ideas and making a general purpose shrinkwrap verification tool, which isn't coupled to shrinkpack.

5id commented 8 years ago

@JamieMason just fyi, I investigated how hard it would be to do this. Fortunately, all remote packages already have a shasum that gets returned in the package json so all we need to do is calculate the sha of the file.

NPM does appear to do a shasum check against the tar when it installs a package - when using shrinkwrap it doesn't seem to do the same check. At least, I replaced one tar with another one and it still said it installed.

The commit in question is over here: https://github.com/5id/shrinkverify/commit/38a0ebce3f3ce2c507bddc3c3bf0d59052655d54

If it's a bit out of scope for this package, I may look at furthering this myself as currently we can't be 100% confidence that the files we have are the same as the files that we would get if we downloaded remotely.

The way I'm planning to use this is something like:

  1. Pull repo onto CD with node_shrinkpack
  2. Re-run shrinkpack (or new tool) to verify packages are properly set/correct
  3. Copy the node_shrinkpack across to a docker image, publish that to a private repo
JamieMason commented 8 years ago

I'm away now until the weekend but I'll have a think whether to add this. I need to understand it better which your last comment will help with when I get back.