rotationalio / honu

Adaptive consistency replication with reinforcement learning for large scale globally distributed storage.
BSD 3-Clause "New" or "Revised" License
12 stars 2 forks source link

Changes for Successful for Anti-Entropy Replication #15

Closed bbengfort closed 2 years ago

bbengfort commented 2 years ago

This PR implements several changes to ensure that Honu can be used effectively for anti-entropy replication including:

Update Method

Anti-Entropy requires the ability to put objects directly to the database without version information being modified. E.g. a repair means placing the version as it was from the remote locally. If the version information was modified during replication then the replication itself would cause updates that would keep bouncing back and forth.

Note that this means we're ending up with method pairs for user interactions and replica interactions: Get/Object, Put/Update - we'll probably also want Delete/Destroy. The user actions: Get, Put, and Delete modify version information while the replica actions Object, Update, and Destroy change the underlying state of the database not just the object.

Version helpers

The first version of an object is {"pid": P, "version": 1, "parent": null}; however replicas need to say they don't have an object, requiring a zero version, which can either be an empty or zero-valued struct or nil.

Bidirectional Streaming Gossip RPC

The original Gossip RPC in the Replication service was a unary RPC; however bilateral anti-entropy requires both a push and a pull. In the unary mechanism, not only is pagination required, but also a secondary request to do the pull (along with trickier state tracking). Refactoring to a bidirectional stream not only improves performance but also prevents the need for pagination and simplifies the anti-entropy implementation.

This PR blocks https://github.com/trisacrypto/directory/pull/280

codecov[bot] commented 2 years ago

Codecov Report

Merging #15 (514eb82) into main (7ab6fd1) will increase coverage by 0.24%. The diff coverage is 60.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #15      +/-   ##
==========================================
+ Coverage   54.13%   54.37%   +0.24%     
==========================================
  Files           9        9              
  Lines         460      480      +20     
==========================================
+ Hits          249      261      +12     
- Misses        173      177       +4     
- Partials       38       42       +4     
Impacted Files Coverage Δ
honu.go 45.60% <55.55%> (+1.67%) :arrow_up:
object/object.go 47.61% <100.00%> (+5.51%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 7ab6fd1...514eb82. Read the comment docs.