search

subconsciousnetwork / noosphere

Noosphere is a protocol for thought; let's discover it together!
Apache License 2.0
668 stars 40 forks source link

Implement CAR Mirror for more efficient block synchronization #20

Open cdata opened 2 years ago

cdata commented 2 years ago

CAR Mirror is a mechanism being researched at Fission for determining the probabilistic minimal bundle of blocks to transfer changes from a source (our clients) to a sink (a receiving gateway).

The spec can be found at https://github.com/fission-codes/spec/blob/main/car-pool/car-mirror/SPEC.md

Right now, we implement a very naive, sparse block bundling scheme that relies on versioning information recorded on each revision to a Sphere to determine which blocks need to be transferred to a gateway. This is fine as a first pass, but it loses its sparseness as soon as we cross the Sphere->Content boundary because we do not currently track discrete revision modifications to content.

CAR Mirror will allow us to decouple sparse bundling of blocks for transit from our conflict-resolution scheme, and also extend bundling sparseness qualities into the content sub-domain of the Sphere data structure.

cdata commented 2 years ago

Fission has been building a Go implementation here: https://github.com/fission-codes/go-car-mirror