npm / cacache

npm's content-addressable cache
Other
279 stars 31 forks source link

[FEATURE] support multiple integrity algorithms #25

Closed isaacs closed 1 year ago

isaacs commented 5 years ago

What / Why

Right now, if you provide multiple algorithms to content.write(), it'll error out with:

opts.algorithms only supports a single algorithm for now

It's said that for a long time. Let's support multiple algorithms!

This causes some suboptimal caching in make-fetch-happen, because we may have an integrity value that is a sha512, but it always caches as sha1, so we can never have a cache hit.

How

In lib/content/write.js, we always place the content in a single location based on the integrity and algorithm.

Current Behavior

isaacs commented 5 years ago

To investigate: the cache index stores a single integrity string, but would have to store multiple. Ie, instead of "integrity":"sha512-deadbeef" it might be "integrity":"sha512-deadbeef sha1-bad1dea". Fall back to secondary locations on ENOENT errors, if multiple possible content addresses are available.

isaacs commented 5 years ago

Might be good to do this along with the flat folders PR, since it's a change to the content/index layout algorithm, so we don't have to bump that twice.

mjsir911 commented 3 years ago

The cache index files seem to support multiple integrity lines per file eg:

4577d1102461b247feed642939cfaba3270dc6e6    {"key":"pacote:tarball:file:/var/tmp/portage/www-plugins/musescore-downloader-0.11.2/distdir/browserify-rsa-4.0.1.tgz","integrity":"sha512-+YpEyaLDDvvdzIxQ+cCx73r5YEhS3ANGOkiHdyWqW4t3gdeoNEYjSiQwntbU4Uo2/9yRkpYX3SRFeH+7jc2Duw==","time":1605029163847,"size":5999}
98128ac4ec588767de288fb67d666ea9c249ac5c    {"key":"pacote:tarball:file:/var/tmp/portage/www-plugins/musescore-downloader-0.11.2/distdir/browserify-rsa-4.0.1.tgz","integrity":"sha1-IeCr+vbyApzy+vsTNWenAdQTVSQ=","time":1604617124075,"size":5999}

which I've been using as a workaround for this issue, and doesn't require changing the integrity string requirements