search

ipfs / kubo

An IPFS implementation in Go
https://docs.ipfs.tech/how-to/command-line-quick-start/
Other
16.18k stars 3.01k forks source link

Work to ensure maximum reproducibility of leaf blocks #2099

Closed btrask closed 7 years ago

btrask commented 8 years ago

This issue is related to #2053 (and maybe #2067?).

There are several different importers, chunkers and layouts possible when importing large files. While IPFS allows considerable flexibility in the merkledag block structure (#1953), it should at least try to ensure that the leaf nodes are always uniform, so that if nothing else they can be de-duplicated. This also allows more nodes to host common blocks, giving better file transfer performance (and a host of other advantages).

whyrusleeping commented 7 years ago

We now have a --raw-blocks flag to add that should help with this, as well as rabin fingerprinting.

Other than those, there isnt anything else actionable here so i'm going to close this (If theres anything actionable, please go ahead and open a new issue)