lnx-search / datacake

Tooling for creating your own distributed systems.
MIT License
390 stars 22 forks source link

Avoid sending duplicate RPC calls during the background distribution task #44

Open ChillFish8 opened 1 year ago

ChillFish8 commented 1 year ago

Currently, datacake's eventual consistency module will send operations in batches to all nodes in the background, but this is in efficient when we're sending operations that a node might have already replied to and acknowledged.

Improving this cuts out some additional RPC calls in situations with a small number of nodes:

3 nodes with Consistency::Quorum will result in at least 3 RPC calls (1for the quorum, 2 for the background task) this can be easily made into 2 and this scales as the consistency level is increased and the number of nodes increases.