hydro-project / fluent

A data-driven compute platform
Apache License 2.0
1.22k stars 173 forks source link

Exposing set lattices #61

Closed cw75 closed 5 years ago

cw75 commented 5 years ago

Now the KVS support both LWW lattice and Set lattice. When submitting a request, the client need to specify the key's lattice type. Note that lattice type is a required field. Otherwise protobuf will fail during the serialization. example:

KeyRequest req;
KeyTuple* tp = req.add_tuples();
tp->set_key(key);
tp->set_lattice_type(kLWWIdentifier);

If we want set lattice, use kSetIdentifier instead.

The server keeps track of the lattice types of existing keys in the KVS. If there is a mismatch between the key's lattice type in the request and in the KVS, the request will be dropped with an error message "Lattice type mismatch".

I ended up changing non-trivial amount of code...mainly because previously some code was customized for the LWW case. With the current code, adding support for more lattice types should be way simpler than before. All tests should be passing but there might be bugs that I haven't found. Let's take a closer look when we start to extensively use a mix of lattice types.

I didn't touch the python client, but we can follow the example in the cpp client to specify the lattice type of a key's value.

codecov[bot] commented 5 years ago

Codecov Report

Merging #61 into master will increase coverage by 1.8%. The diff coverage is 66.33%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master      #61     +/-   ##
=========================================
+ Coverage   62.48%   64.28%   +1.8%     
=========================================
  Files          50       50             
  Lines        1458     1596    +138     
=========================================
+ Hits          911     1026    +115     
- Misses        547      570     +23
Impacted Files Coverage Δ
kvs/src/route/seed_handler.cpp 100% <ø> (ø) :arrow_up:
kvs/src/kvs/rep_factor_change_handler.cpp 5.55% <0%> (+2.77%) :arrow_up:
kvs/src/hash_ring/hash_ring.cpp 0% <0%> (ø) :arrow_up:
kvs/src/kvs/gossip_handler.cpp 6.81% <0%> (-3.44%) :arrow_down:
kvs/src/kvs/rep_factor_response_handler.cpp 1.56% <1.85%> (+0.66%) :arrow_up:
kvs/tests/kvs/test_node_join_handler.hpp 100% <100%> (ø) :arrow_up:
kvs/src/kvs/self_depart_handler.cpp 54.05% <100%> (ø) :arrow_up:
kvs/include/kvs/lww_pair_lattice.hpp 100% <100%> (ø)
kvs/tests/kvs/server_handler_base.hpp 100% <100%> (ø) :arrow_up:
kvs/src/route/replication_response_handler.cpp 44% <100%> (+8.58%) :arrow_up:
... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 8cdff6b...bfa8af8. Read the comment docs.