Open michaelmaguire opened 7 years ago
See Gavin Lowe for paper on Linearizability Testing
http://www.cs.ox.ac.uk/people/gavin.lowe/LinearizabiltyTesting/paper.pdf
See:
https://github.com/jepsen-io/knossos
for implementation
Timeline diagram reads from top -> bottom in time:
http://ctolab.xtremeghost.com/student32/jepsen/files/zookeeper/20161214T110856.000Z/timeline.html
Failure of linearity shown in: http://ctolab.xtremeghost.com/student32/jepsen/files/zookeeper/20161214T110856.000Z/linear.svg
which reads left -> right in time.
Sample Zookeeper logs:
admin@ip-192-168-32-100:~/jepsen.zookeeper master$ ssh w5 less /var/log/zookeeper/zookeeper.log
2016-12-13 14:54:32,162 - INFO [main:QuorumPeerConfig@101] - Reading configuration from: /etc/zookeeper/conf/zoo.cfg
2016-12-13 14:54:32,169 - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
2016-12-13 14:54:32,169 - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
2016-12-13 14:54:32,169 - INFO [main:DatadirCleanupManager@101] - Purge task is not scheduled.
2016-12-13 14:54:32,170 - WARN [main:QuorumPeerMain@113] - Either no config or no quorum defined in config, running in standalone mode
2016-12-13 14:54:32,191 - INFO [main:QuorumPeerConfig@101] - Reading configuration from: /etc/zookeeper/conf/zoo.cfg
2016-12-13 14:54:32,199 - INFO [main:ZooKeeperServerMain@95] - Starting server
2016-12-13 14:54:32,211 - INFO [main:Environment@100] - Server environment:zookeeper.version=3.4.5--1, built on 03/16/2014 21:36 GMT
2016-12-13 14:54:32,211 - INFO [main:Environment@100] - Server environment:host.name=w5
2016-12-13 14:54:32,211 - INFO [main:Environment@100] - Server environment:java.version=1.8.0_111
2016-12-13 14:54:32,212 - INFO [main:Environment@100] - Server environment:java.vendor=Oracle Corporation
2016-12-13 14:54:32,212 - INFO [main:Environment@100] - Server environment:java.home=/usr/lib/jvm/java-8-oracle/jre
2016-12-13 14:54:32,212 - INFO [main:Environment@100] - Server environment:java.class.path=/etc/zookeeper/conf:/usr/share/java/jline.ja
r:/usr/share/java/log4j-1.2.jar:/usr/share/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/netty.jar:/usr/share/ja
va/slf4j-api.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/zookeeper.jar
2016-12-13 14:54:32,212 - INFO [main:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/l
ib64:/lib:/usr/lib
2016-12-13 14:54:32,212 - INFO [main:Environment@100] - Server environment:java.io.tmpdir=/tmp
2016-12-13 14:54:32,212 - INFO [main:Environment@100] - Server environment:java.compiler=<NA>
2016-12-13 14:54:32,214 - INFO [main:Environment@100] - Server environment:os.name=Linux
2016-12-13 14:54:32,214 - INFO [main:Environment@100] - Server environment:os.arch=amd64
2016-12-13 14:54:32,214 - INFO [main:Environment@100] - Server environment:os.version=3.16.0-4-amd64
2016-12-13 14:54:32,215 - INFO [main:Environment@100] - Server environment:user.name=zookeeper
2016-12-13 14:54:32,215 - INFO [main:Environment@100] - Server environment:user.home=/var/lib/zookeeper
2016-12-13 14:54:32,215 - INFO [main:Environment@100] - Server environment:user.dir=/
2016-12-13 14:54:32,224 - INFO [main:ZooKeeperServer@726] - tickTime set to 2000
2016-12-13 14:54:32,224 - INFO [main:ZooKeeperServer@735] - minSessionTimeout set to -1
2016-12-13 14:54:32,224 - INFO [main:ZooKeeperServer@744] - maxSessionTimeout set to -1
2016-12-13 14:54:32,253 - INFO [main:NIOServerCnxnFactory@94] - binding to port 0.0.0.0/0.0.0.0:2181
2016-12-13 14:54:32,272 - INFO [main:FileTxnSnapLog@240] - Snapshotting: 0x0 to /var/lib/zookeeper/version-2/snapshot.0
Examples of Zookeeper "Linearizable" fails (using test code in this repo)
http://ctolab.xtremeghost.com/student7/jepsen/files/noop/20161214T112120.000Z/linear.svg
http://ctolab.xtremeghost.com/student7/jepsen/files/noop/20161214T112120.000Z/timeline.html
Another stale read example:
http://ctolab.xtremeghost.com/student4/jepsen/files/zookeeper/20161214T110938.000Z/timeline.html
Zookeeper claims to offer "Linearizable Writes" with "Sequential Reads" consistency:
( "strongest" at the top, "weaker" at the bottom. https://aphyr.com/posts/313-strong-consistency-models )
It does not offer "Linearizable" or "Strong Serializable".
For fun, to illustrate the Jepsen framework, we tested Zookeeper against a higher level of consistency, namely "Linearizable", expecting it to fail.
Creating a test framework:
admin@ip-192-168-32-100:~$ lein new jepsen.zookeeper
Then edit
jepsen.zookeeper/project.clj and jepsen.zookeeper/src/zookeeper.clj
as show in this project's source.
Running a test:
admin@ip-192-168-32-100:~/jepsen.zookeeper master$ lein run test --nodes-file ~/nodes --username admin | tee output.log