Open Jorropo opened 2 years ago
Cc @rvagg @dignifiedquire that Kubo maintainers are going to take the first stab at getting this written in September. Feel free to watch or leave any notes.
Our latest set of trials & tribulations from Iroh: https://github.com/n0-computer/iroh/pull/198 and our running doc of papercuts: https://number-zero.notion.site/UnixFs-742339892d9c47d5b79f4f942e661bbf
@b5 about https://github.com/n0-computer/iroh/pull/198 I think balanced tree is not in the spec. Or at least, if someone really care about it, it's a non authoritative part of the spec.
As long as you get your file sizes rights, and the merkle dag is correct (mean that a correctly build decoder successfully rebuild the original content). You can use whatever scheme you like.
Sure, maybe not an authoritative part of the spec, but as Lidel pointed out in the implementers call yesterday, there are many things that would be good to suggest within spec documents that give implementers hints so the don't footgun themselves.
No one says the dag needs to be balanced. Everyone ends up implementing a balanced tree at some point.
Some additional asks, based on real world problems I've seen:
Tsize
means in context of UnixFS and non-UnixFS sub-DAGs (total size, including all IPFS/IPLD envelopes)
Another ask is having a recommendation for how to add non-canonical / extensions to the spec e.g. systematization for extra metadata
I want to do this, but as a followup IPIP.
First I start by describing what Kubo do without any new inovation, then we see what we can improve. It will be impossible to review if we mix and match old and new stuff.
Here is the in-progress PR: https://github.com/ipfs/specs/pull/331
We need:
Tsize
(total subdag size, raw data + envelopes) and raw file data (without IPFS metadata), and how to read /interpret each.