tristanls / gossipmonger

Gossip protocol endpoint for real-time peer-to-peer replication
MIT License
51 stars 10 forks source link

Deleting keys #4

Closed webglider closed 9 years ago

webglider commented 9 years ago

Is there any way to safely delete keys?

tristanls commented 9 years ago

Hi,

I am not exactly sure what you mean by safely and what you mean by delete :smile:

The simple answer is (based on my assumptions of what you mean), in order to safely delete, you should mark the key with a "tombstone" that lets the reader know that it is deleted.

So, for example, if you previously set a key to some value:

gossipmonger.update('foo', 'bar');

Later, if you want to delete it, you should update it to some "tombstone" value:

gossipmonger.update('foo', 'there-is-no-way-this-is-a-real-key-only-deleted-keys-will-have-this-value');

The less simple answer...

Gossipmonger implements scuttlebutt gossip protocol. It turns out that vector-clock scuttlebutt gossip is a special case of a delta-mutation CRDT. The paper where I first learned about delta-mutation CRDTs is here http://arxiv.org/abs/1410.2803 . What this means to me, is that any system using scuttlebutt gossip, is in effect, a CRDT. The "problem" with CRDTs is that they have to maintain the entire history of everything that has ever happened in order to maintain all the property guarantees of a CRDT. Logical deletes are possible, but they're more involved, with the tombstone example above demonstrating a delete like one would do in a Last-Writer-Wins Register (LWW-Register)(in this case, since each node is a single writer, there is no timestamp conflict). This means that in order to logically delete data, one must add historical data to the CRDT. Practically, you could choose to truncate the history that CRDT is maintaining, by using some sort of timeout, however, at that point, all of the gossip and CRDT guarantees of the distributed system are lost.

webglider commented 9 years ago

@tristanls Thanks for the detailed reply :)