search

ipfs-inactive / archives

[ARCHIVED] Repo to coordinate archival efforts with IPFS
https://awesome.ipfs.io/datasets
183 stars 24 forks source link

Independently verify downloaded datasets #125

Open ghost opened 7 years ago

ghost commented 7 years ago

Some datasets may contain hashes/checksums of the included files, which makes it easy to verify the integrity of the downloaded files. For datasets which don't, it's useful to download them again somewhere else, and compare the ipfs hashes.

This implies that downloaders sign and publish the ipfs hashes of the stuff they've finished downloading.

mguentner commented 7 years ago

You could use git-annex for that: http://git-annex.branchable.com/ It's already tested for large archives and could be used if the IPFS import needs to be repeated in the future. This could also serve as a basis for the initial download processing.