ipld / specs

Content-addressed, authenticated, immutable data structures
Other
592 stars 108 forks source link

Endianness in IPLD Codecs #321

Open vmx opened 3 years ago

vmx commented 3 years ago

It's important for content-addressing to have the same endianness for We should either have a generic rule or have every codec should have a section about endianness.

I propose that we write in a general document for all codecs that endianness is expected if data is serialized as fixed sized integers or floats. Varints are a special case. I consider them being their own type, but effectively they are big-endian (though I'm not sure if their big-endianness should be spell that out explicitly).

rvagg commented 3 years ago

I think that this probably belongs in the codec docs themselves rather than a dedicated doc. It's quite different from something like UTF-8 questions that get into in-memory and API issues. Endianness just needs clarity when converting to and from bytes and so far we get to rely on upstream specs, but as we work more on our own codecs then this absolutely needs to be in those. I'm just not sure it rises to the level of needing its own doc?

vmx commented 3 years ago

I'm just not sure it rises to the level of needing its own doc?

I rather meant "top-level" doc than "own doc" in the sense that if it turns out to be the same for all our codecs, then we can mention it at a single place spanning all codecs, rather then putting into each of them. But that's just a minor thing, we can do whatever works best :)