Violations of causal consistency found

[SuperLeilia]

Hi, we have found a bug while checking causal consistency of Antidote. We have printed operations to the database and return values from database into logs.

The configurations are listed below:

• AntidoteDB 0.2.2
• 6 clients (3 clients per maachine), 20 transactions peer clients, 20 operations per transactions with 5 keys

The log file is attached at the end of this post . Note that operation is in the type of operation_type(variable, value, client_id, transaction_id).

We have found that transaction 0 can reach transaction 39 by the order in each client and write-read relation. Aiso in transaction 39 we have found w(0,62) and in transaction 0 we have found r(0,62), so transaction 39 can reach transaction 0, which made it a cycle. Obviously it violates causal consistency.

Datalog files FYI: result.txt

[bieniusa]

Thank you @SuperLeilia! We will try to reproduce the setup and test to debug. One question: What type of CRDT are you using for the reads/writes, e.g. LWWRegister?

[SuperLeilia]

Hi @bieniusa , I used LWWRegister and with Python client.

[SuperLeilia]

To facilitate your debugging, I attached my test code @bieniusa

data_generator_antidote.txt

[peterzeller]

The Antidote Python client currently does not support session guarantees, so there is no happens-before relation between operations that are executed on the same thread. Does your analysis depend on session guarantees or is it still a consistency violation?

Session guarantees are supported in the Javascript client (see https://antidotedb.github.io/antidote_ts_client/#session-guarantees) and Java client (pass CommitInfo timestamp to a transaction).

[SuperLeilia]

Thanks for your reply, @peterzeller ! Some comments/questions:

1. We followed https://github.com/AntidoteDB/antidote-python-client, as well as the official document, to deal with client-side logic (for txns). We couldn't find anywhere that python clients cannot work with causal consistency. Are we missing anything?

2. What do you mean by "no happens-before relation"? Our ops and txns are closed loop and they are definitely ordered by the program order, right? Additionally, based on the returned data, we can infer the write-read order among operations/txns (from possibly different clients) as well, right? If Antidote provides causal consistency, we shouldn't have seen the counterexample posted above.

3. For python client, what about causal consistency at the process level (e.g., one single thread per process)? There should not be any doubt that a process itself constitutes a session, right?

4. Are you suggesting that, to benefit from Antidote's causal consistency guarantee, we must currently use either Javascript or Java client?

Looking forward to your reply! Thanks!

[peterzeller]

@SuperLeilia I have added the option to the python client in this PR: https://github.com/AntidoteDB/antidote-python-client/pull/4

With this change you can set the minimum snapshot time in your code like this:

tx = clt.start_transaction(min_snapshot=clt.last_commit)

The last_commit field is set whenever you have committed a transaction.

Our ops and txns are closed loop and they are definitely ordered by the program order, right?

Antidote does not use the implicit program order - you need to pass the commit timestamp from the previous operation to the next operation manually.

I don't remember exactly why we designed it like this, but I think it was something like this:

1. Ideally, Antidote clients would automatically fail over to different data centers if the current one cannot be reached. If you always want to enforce program-order that would mean a potential very long wait until the updates from the crashed data center are replicated to the other one. With explicit ordering the programmers have more control to state where the order is important and where it is not.
2. In many programming languages, program order is not so easy to define. With thread-pools and features like async/await you can have code that looks like it happens on one thread but actually happens on several threads.

Would be cool if you could try with the min_snapshot option.

[SuperLeilia]

@peterzeller Thanks for your reply! I noticed that the new commits (e.g. last_commit field) are on the branch 'session-guarantees-support'. We install Antidote client via "pip antidotedb" which we assume works with the master branch. How can we proceed with the testing?

[bieniusa]

It is on master now; I just merged the PR.

[preguica]

I have uploaded the current version to pip... you should be able to use the new version using pip now.

Annette Bieniusa @.***> escreveu no dia domingo, 30/01/2022 à(s) 12:23:

It is on master now; I just merged the PR.

— Reply to this email directly, view it on GitHub https://github.com/AntidoteDB/antidote/issues/458#issuecomment-1025132062, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFD4TDE5465ALKA3B5X5DUYUUSTANCNFSM5JEOXA4A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

[20211202na]

@preguica @bieniusa @peterzeller Thanks for fixing the python client issue!! The new version is running smoothly so far. I'll get back to you if any anomaly identified :D