search

ipld / js-car

Content Addressable aRchive format reader and writer for JavaScript
Other
46 stars 7 forks source link

feat: add carv2-messaging example for embedding messages in CARv2 #89

Open rvagg opened 2 years ago

rvagg commented 2 years ago

Background

This comes from a feature request by @mikeal to be able to signal additional, potentially arbitrary properties for backend communication, like supplying "PieceCID" for the CAR, and other things .. pushing CAR from just a block transport to also have a messaging layer that relates to those blocks that's more than just "here are the roots".

Proposal

So this is .. sneaky .. we take a standard CARv1 (in this case, just a single-block simplistic example), wrap it in a CARv2, push the "data offset" back enough to embed a length-prefixed dag-cbor "message" just after the CARv2 header.

We have a "characteristics" bitfield in CARv2 but only one bit that's currently being used, this proposes that the second-to-leftmost bit being used to indicate "message after v2 header".

The resulting .car file is readable by js-car as-is, the API shows you what you'd see if you passed it the original CARv1 contents, but it's wrapped in a CARv2 and you can poke at the bytes to see that there's a secret message in there just after the CARv2 header and before the CARv1 payload.

It's also valid according to go-car:

$ car list example-messaging.car 
bafkreihwkf6mtnjobdqrkiksr7qhp6tiiqywux64aylunbvmfhzeql2coa
$ car inspect example-messaging.car 
Version: 2
Characteristics: 40000000000000000000000000000000
Data offset: 158
Data (payload) length: 120
Index offset: 0
Index type: (none)
Roots: bafkreihwkf6mtnjobdqrkiksr7qhp6tiiqywux64aylunbvmfhzeql2coa
Root blocks present in data: Yes
Block count: 1
Min / average / max block length (bytes): 24 / 24 / 24
Min / average / max CID length (bytes): 36 / 36 / 36
Block count per codec:
        raw: 1
CID count per multihash:
        sha2-256: 1

I'm interested to see who might object to such flagrant abuse of CARv2 flexibility ..

rvagg commented 2 years ago

I should also say - this is very manual in here, if we adopt this then the API would have to be made to handle at least some of this so you don't have to get so dirty with the bytes and bits... cause it's kind of nasty atm.

rvagg commented 2 years ago

go-car impl (including ability to read and recreate the .car created by this example): https://github.com/ipld/go-car/pull/322