search

orbitdb-archive / ipfs-log

Append-only log CRDT on IPFS
https://orbitdb.github.io/ipfs-log/
MIT License
398 stars 55 forks source link

Feat/ipfs dag #210

Closed shamb0t closed 5 years ago

shamb0t commented 5 years ago

This PR updates ipfs.object calls to use ipfs.dag API

shamb0t commented 5 years ago

Some concerns discussed in #200

haadcode commented 5 years ago

Looking really good @shamb0t! 👍 Would it be possible to split this PR into two: Feat/ipfs-dag and "Update webpack and babel" (which only upgrades the deps as you've done here)?

satazor commented 5 years ago

We are not using the links feature of ipld. The main advantage is that we can use ipld explorer or even use path queries.

This means that the next array on entries and the heads array on log entries should be ipld links. I already had this implemented locally but never made a PR due to the slowness but we now know what was the cause :p.

satazor commented 5 years ago

Also regarding this, we can either use cbor or protobuf. cbor is easier to do because we just need to write a json object like so:

ipfs.dag.put({
   ...e,
   next: e.next.map((cid) => ({ '/': cid })
});

While using pb you have to construct the dagNode manually using https://github.com/ipld/js-ipld-dag-pb in order to specify ipld links. Protobufs should be faster than cbor but I'm not really sure, perhaps @vmx could advise us here.

haadcode commented 5 years ago

@satazor would you want to PR the above "use IPLD links"? We define the shape of the entry objects here: https://github.com/orbitdb/ipfs-log/blob/master/src/entry.js#L30.

aphelionz commented 5 years ago

Does this approach have the effect described here? https://github.com/orbitdb/ipfs-log/issues/106#issuecomment-402950785

the problem with this is that as far as I know there is no way to tell IPFS to not resolve a Link. This means reading an Oplog HEAD via IPFS would cause it to try to return the whole Oplog or something like that.

vmx commented 5 years ago

perhaps @vmx could advise us here.

My advise is to use dag-cbor and not using dag-pb. I hope that one day dag-pb will not be used anymore. If things don't work/are slow it should be fixed, rather then having someone using dag-pb.

satazor commented 5 years ago

@haadcode Yep, and as per @vmx advice we should use cbor. I will be giving a workshop in a few hours so it’s difficult me for me to focus on this, but I already had everything using ipld links using cbor. I can either open a new PR with that or let @shamb0t change this PR instead.

haadcode commented 5 years ago

@satazor if you have the code ready, please do PR it and whoever can then cherry pick the approriate commits/changes to this PR 👍

shamb0t commented 5 years ago

Thanks for the info here guys! @satazor Feel free to open a new PR with cbor ipld links whenever you're ready and we can work from there

satazor commented 5 years ago

Alright, I will do that either tonight or earlier in the morning. @shamb0t feel free to take on this if you want and have the time! (I'm in transit)

shamb0t commented 5 years ago

closing in favour of #213