Set up Jepsen test for Dgraph

dgraph-io / dgraph

The high-performance database for modern applications

https://dgraph.io

Other

20.42k stars 1.5k forks source link

Set up Jepsen test for Dgraph #711

Closed manishrjain closed 6 years ago

manishrjain commented 7 years ago

Aphyr is super busy -- so it would be a while before he tests Dgraph. We have a few things that need to go in, but we're like to at least do a prelim test to find any edge cases that we haven't handled. So, would be great if someone could run Jepsen test on Dgraph (talk to us before, so we can put those last 5% things in).

foobatman commented 7 years ago

Do we already have a test suite for jepsen ? Or does this involve creating one ?

manishrjain commented 7 years ago

Nope, got no idea about what it involves to run Jepsen; so would require setting one up.

foobatman commented 7 years ago

Okay. Nice area for me to start helping dgraph. Will start looking at it.

On Fri, Mar 17, 2017, 12:27 PM Manish R Jain notifications@github.com wrote:

Nope, got no idea about what it involves to run Jepsen; so would require setting one up.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/dgraph-io/dgraph/issues/711#issuecomment-287281450, or mute the thread https://github.com/notifications/unsubscribe-auth/AB-j0RoZLCvAVnx9-TPKdwLmj1daLvADks5rmi7qgaJpZM4MgJkH .

foobatman commented 7 years ago

Can you tell me few guarantees that you expect Dgraph to always satisfy when running in clustered environment ? One that I can think of is linearizability. Initially, it would be good to start asserting on those and then add more edge cases. What does hivemind reckon about it ?

foobatman commented 7 years ago

Also I find that Dgraph's architecture is somewhat closer to https://github.com/cockroachdb/cockroach. This link ( https://jepsen.io/analyses/cockroachdb-beta-20160829#fnref1 ) shows jespen analysis for it. Can we move on similar lines ?

manishrjain commented 7 years ago

I think linearizability is one of the top ones.
Then, avoiding data loss in case of crashes.
Bringing up a new replica server, and ensuring that its version of data equals other replicas.

Also, we intend to use full linearizability provided by Raft; unlike how CockroachDB does it.

petersondrew commented 6 years ago

I know I'm bumping a very old issue here, but has there been any work done on running DGraph under Jepsen?

manishrjain commented 6 years ago

Yes, Kyle himself is working on this :-).