Sync is network-bound, and the private partition evenly distributes data by design. While we're waiting for GraphSync to be production ready, the easily solution to improving performance is to flatten the structure. A 1024-weighted HAMT (i.e. 128-byte bitmask headers) hits a sweet spot efficiently getting a node to queryabile sync in few round trips and overhead.
expede
commented
2 years ago
https://whitepaper.fission.codes/file-system/partitions/private-directories/data-layer#secure-content-prefix-tree
Especially with #301 (immutable core), the
Sync is network-bound, and the private partition evenly distributes data by design. While we're waiting for GraphSync to be production ready, the easily solution to improving performance is to flatten the structure. A 1024-weighted HAMT (i.e. 128-byte bitmask headers) hits a sweet spot efficiently getting a node to queryabile sync in few round trips and overhead.