hydro-project / anna

A low-latency, cloud-native KVS
Apache License 2.0
702 stars 106 forks source link

Replication mechanism in Anna #25

Closed authwork closed 4 years ago

authwork commented 4 years ago

I have some questions about the replication mechanism in Anna:

  1. Is there any document talking about how to sync all replicas of a key-value pair? or the workflow of how anna receivers a user PUT request and distributed it to all replicas (N replicas) and how anna receivers a user PUT request and distributed it to 1/K replicas out of all replicas (N replicas)?
  2. How to disabled Anna’s elasticity mechanism? Can I just not start anna-monitor?
vsreekanti commented 4 years ago
  1. Currently, the easiest way to sync all replicas is to use the get_all client method, which retrieves data from all replicas and merges them. In general, requests are routed to an individual KVS node. We use a background gossip/multicast protocol to asynchronously send updates from the receiving replica to all the other replicas on the hash ring responsible for the key. You can find a more detailed description of the protocol in the original Anna paper.
  2. You can disable the various scaling mechanisms in the Anna config.
authwork commented 4 years ago

Many thanks for you help!

authwork commented 4 years ago

Hello! @vsreekanti After reading get_all client method, I have one question. Given a set k = {v1, v2, v3, v4} and it has three replications R1(k) = {v1, v2, v3, v4}, R2(k) = {v1, v2, v3, v4}, R3(k) = {v1, v2, v3, v4}. I execute two command: Delete v4 Add v5 and we know the correct consistent answer is k = {v1, v2, v3, v5}. However, because of the EC, R1(k) = {v1, v2, v3}, R2(k) = {v1, v2, v3, v5}, and R3(k) = {v1, v2, v3, v4}. Using get_all client method will read R1(k), R2(k), and R3(k), then merge their values (Union):

void merge(const Lattice<T> &e) { return do_merge(e.reveal()); }

void do_merge(const set<T> &e) {
    for (const T &elem : e) {
      this->element.insert(elem);
    }
  }

So, the answer is {v1, v2, v3, v4, v5}, which is not the correct consistent answer.