Aggregation --> CRDTs discussion

[davidar]

So, a lot of the questions about "dynamic content on ipfs" seem to boil down to how to aggregate content from multiple sources. Use cases:

• forums (also see zeronet)
• distributed search engine (ipfs/archives#8)
• wikis
• a single dataset partitioned across multiple nodes, with each node only having permission to push updates to their own section
• other stuff, probably...

This should be possible with IPFS+IPNS, but it needs to be well-documented and streamlined for users.

Another issue is how to make it scale, so you're not having to aggregate millions of sources in every client (e.g. reddit on ipfs).

[daviddias]

Hi y'all, some of us gathered today to discuss orbit's future, essentially gather ideas of what is possible.

Some notes from the discussion:

• orbitdb today offers:
  • document store
  • kv store
  • feed
• orbitdb uses ipfs-log for its op log (which is a CRDT in itself)
• Investigate CRDT implementations in JS (yjs, swarmjs, gun, ssb, hyperlog and so on), identify its features and limitations.
• offer the same levelDOWN interface on top of orbit and explore what that brings to the levelDB ecosystem
• collect all the CRDT research in one place

We've also realized that since CRDTs are actually a broad topic with a lot of people interested, that it would be better to open a repo research-CRDT and have the discussions and gather of materials there. You can find it here: https://github.com/ipfs/research-CRDT

[ianopolous]

I would love to get the opinion of some CRDT experts here on what I believe is a new design for a Set CRDT: https://github.com/ianopolous/crdt It has better semantic properties than a PN-set (you cannot reach a situation where an add operation doesn't end in the element being present) and better storage size scaling than an OR-Set and is very simple.

[pgte]

In the context of the IIIF-over-IPFS project, I created an IPFS connector for Yjs (a CRDT library). This allows any Y.js CRDTs to be exchanged from any in-browser peer.

Demo (using a IIIF data structure) here.

Next steps:

• create an IPFS store for Y.js. Currently, Y.js supports IndexedDB, Leveldb and memory. The plan is to abstract IPFS to provide similar primitives, reusable in other contexts.
• take these learnings and apply them to other CRDT libraries, like swarm.js and others.

Links:

• Yjs
• y-ipfs-connector

[daviddias]

Just came here to say that now there is a awesome video tutorial of how to get Yjs to use IPFS!

https://github.com/ipfs/research-CRDT/issues/7

❤️👏🏽👏🏽 Thanks @pgte 👏🏽👏🏽💖

[alishoker]

@ianopolous Have you seen this ORSet? http://haslab.uminho.pt/ashoker/files/deltacrdt_0.pdf

[ElMehdiBouamama]

While reading the multiple comment posted out here, a question came up to my mind.

• Does anyone taught of implementing the hashgraph technologie over IPFS?
• Does the hashgraph technologie solve in someway the problem of crdt / speed / consensus problem? I am still a student, and have very few knowledge over thoses technologies. So if someone can lighten me up, i would be greatfull 🔨

[beenotung]

Just to add more context, IPFS is open and free, hashgraph is patented in US.