Schultzer
commented
1 month ago Looks like this work has stalled and I’m wondering is improving mnesia’s consistency around network splits regardless of implementation something the OTP team is actively interested in?
Is your feature request related to a problem? Please describe. Mnesia's default partition tolerance model is to leave it to the programmer during a network partition (https://www.erlang.org/doc/apps/mnesia/mnesia_chap7#recovery-from-communication-failure). Moreover Mnesia only provides two access contexts, dirty and transactions, and lacks intermediate apis that provide consistency. For example, dirty operations might leave the replicas in different states depending on the order of the messages being delivered.
Describe the solution you'd like I am proposing a new API for Mnesia, which uses CRDTs for eventual consistency. This provides consistency for asynchronous operations (unlike transactions, which wait for all nodes to complete), but also gives better consistency guarantee than dirty operations. Moreover, this could also be used to add partition tolerance and auto conflict resolution after a network partition, although the exact semantics requires some careful tuning.
It is currently implemented in a fork and I plan to submit PRs shortly. As this prototype is still at an early stage, I would appreciate feedbacks.
Describe alternatives you've considered Alternative solutions might be using read-write quorums, or maybe use Paxos like consensus algorithms for stronger consistency guarantees. But Mnesia inherently works in a leaderless way, and I feel CRDTs are most suited to this style of communication.
Additional context Add any other context or screenshots about the feature request here.
I also recorded a demo on the issues of dirty operations and the functionality of this new API.
Feel free to reach me out if you want to learn more about the technical details of this implementation.