ipfs / npm-kubo

Install Kubo (go-ipfs) from NPM
https://www.npmjs.com/package/kubo
MIT License
41 stars 30 forks source link

feat: cache and verify downloaded archive #32

Closed lidel closed 3 years ago

lidel commented 3 years ago

This PR saves us time on CI and gives more confidence that downloaded archive is valid:

Demo: caching and verifying in action

First time downloads archive to the cache:

Downloading https://dist.ipfs.io/go-ipfs/v0.7.0/go-ipfs_v0.7.0_linux-amd64.tar.gz to /home/lidel/.cache/npm-go-ipfs
Downloaded https://dist.ipfs.io/go-ipfs/v0.7.0/go-ipfs_v0.7.0_linux-amd64.tar.gz
Downloading go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512
Downloaded go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512
Verifying go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512
OK (1d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086)
Unpacked /home/lidel/project/ipfs/npm-go-ipfs
Linking /home/lidel/project/ipfs/npm-go-ipfs/go-ipfs/ipfs to /home/lidel/project/ipfs/npm-go-ipfs/bin/ipfs

Second time reused archive from the cache:

https://dist.ipfs.io/go-ipfs/versions
Found /home/lidel/.cache/npm-go-ipfs/go-ipfs_v0.7.0_linux-amd64.tar.gz
Verifying go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512
OK (1d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086)
Unpacked /home/lidel/project/ipfs/npm-go-ipfs
Linking /home/lidel/project/ipfs/npm-go-ipfs/go-ipfs/ipfs to /home/lidel/project/ipfs/npm-go-ipfs/bin/ipfs

Note that SHA512 is compared on every run. If a single bit was flipped, it will return an error:

Found /home/lidel/.cache/npm-go-ipfs/go-ipfs_v0.7.0_linux-amd64.tar.gz
Verifying go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512
Expected   SHA512: 2d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086
Calculated SHA512: 1d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086
Error: SHA512 of /home/lidel/.cache/npm-go-ipfs/go-ipfs_v0.7.0_linux-amd64.tar.gz' (1d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086) does not match expected value from /home/lidel/.cache/npm-go-ipfs/go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512 (2d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086)
    at cachingFetchAndVerify (/home/lidel/project/ipfs/npm-go-ipfs/src/download.js:67:11)

cc @andrew

lidel commented 3 years ago

@achingbrain as agreed, I've updated tests and removed dead code ported from npm-go-ipfs-dep + added git workflow to run npm test on PRs.

I'm merging this, so the cache is included when 0.9.0 ships.