search

basho / riak_repl

Riak DC Replication
Apache License 2.0
56 stars 32 forks source link

Provide pre/post repl hooks for local/remote representation of objects #322

Open russelldb opened 11 years ago

russelldb commented 11 years ago

With likely larger sets of clusters involved in MDC, it would be very valuable to keep some information within the local cluster only.

An example would be DVVSets / Vector clocks. The pruning of DVVSets depends on the N-Val of the key. Without local/remote cluster views we need to allow N*Clusters actors, with we can have N+Clusters actors. And we can differentiate the local actors from the remote ones so that we only ever prune local actors.

With CRDTs we need to perform garbage collection periodically. With a local / remote view of data, GC can be performed in the local cluster only, rather than across the set of MDC clusters.

I don't have a strong idea about how to do this feature yet, so let's talk hear please.

The rough idea is pre / post repl hooks that allow data types to provide repl with a remote view before being repl'd and local view upon receiving repl'd data.

bookshelfdave commented 11 years ago

See also: https://github.com/basho/riak_repl/issues/305 https://github.com/basho/riak_repl/issues/239

buddhisthead commented 11 years ago

@russelldb Do you need to modify the object before the commit? Or more generically, do you need to be part of the commit process or just get pre- and post-notification? It seems like events (which are suggested in #305) would serve better than the current commit hooks, with event semantics that do not allow for effects on the commit.

cmeiklejohn commented 10 years ago

Moving to the 2.1 release.