search

dominictarr / scuttlebutt

peer-to-peer replicatable data structure
Other
1.32k stars 66 forks source link

Pub/Sub broker on top of Scuttlebutt #31

Open mcollina opened 10 years ago

mcollina commented 10 years ago

I'm thinking about building a distributed pub/sub broker on top of scuttlebutt. Do you think it's a silly idea or it's feasible? Have you got some gut feeling about the performance of such a system?

Another question: it is possible to have sublevel-style gossiping? E.g. limiting the sharing only to a subpart of a model.

dominictarr commented 10 years ago

Yes, I think that is a pretty good idea! The weakness of the scuttlebutt protocol is that you need to do the handshake for each part your replicate. This means that on every subscribe you'd need to send a vector clock for the stuff you already know in that section.

You probably could do something like that, but you'd need to rebuild the vector clock for the subsection /foo/bar that only contained the updates in bar.

This would only be a problem if you where subscribed to /foo/bar and then later decided to subscribe to /foo/* although bar is within that set, a node may have sent an update to bar but also to baz.

dominictarr commented 10 years ago

I think that should be the only edgecase, though.