Running Jepsen tests on a pn-counter with partitioning has a high failure rate.
Please see previous issue for configuration and the environment to reproduce. Then:
lein run test --test-count 20 --workload pn-counter --rate 100 --topology dcs --nemesis partition
Most tests consistently fail most of the time:
all client op's return :ok
most final reads are unequal and most are invalid
non-monotonic reads with grow-only counters
Here's a plot of node read value differences from the complete total order counter, most tests are similar, showing 2 nodes ending up in a consistent state with 3 nodes diverging (swing, from p's to n's, counter):
And a counter where 4 nodes converge to an invalid value:
The nature and comprehensiveness in the respose to partitioning seemed to merit its own issue. Also to raise the status of the current zero mq being used.
Please suggest if it would be helpful to make any changes to the tests or environment.
Running Jepsen tests on a pn-counter with partitioning has a high failure rate.
Please see previous issue for configuration and the environment to reproduce. Then:
Most tests consistently fail most of the time:
:okHere's a plot of node read value differences from the complete total order counter, most tests are similar, showing 2 nodes ending up in a consistent state with 3 nodes diverging (swing, from p's to n's, counter):
And a counter where 4 nodes converge to an invalid value:
The nature and comprehensiveness in the respose to partitioning seemed to merit its own issue. Also to raise the status of the current zero mq being used.
Please suggest if it would be helpful to make any changes to the tests or environment.