Closed nurturenature closed 2 months ago
Client has been updated:
The non-electric SQLite3 GitHub action has been renamed and updated to reflect the tests run successfully when not electrified:
@nurturenature Thanks a lot for sharing these results! We'll be taking a closer look at the anomalies.
👋 we've been working the last month on a rebuild of the Electric server over at a temporary repo https://github.com/electric-sql/electric-next/
You can read more about why we made the decision at https://next.electric-sql.com/about
We're really excited about all the new possibilities the new server brings and we hope you'll check it out soon and give us your feedback.
We're now moving the temporary repo back here. As part of that migration we're closing all the old issues and PRs. We really appreciate you taking the time to investigate and report this issue!
Hi,
I've had a sense that electrified tables were behaving differently than expected. This lead to writing a test that shows non-atomic transactions with intermediate reads are common.
The schema:
Read and append transactions are generated:
The
better-sqlite3
driver is used:electricsql/electric:latest
Docker images and"electric-sql": "^0.10.1"
client are used in a cluster of:Here's an example of an intermediate read:
Running the same test on a non-electrified db shows no anomalies.
The test finds a variety of cycles and anomalies. There are cases where the process will not read its writes or writes are lost. Those will be covered in a future test more suited to illustrate that class of anomalies.
After doing some research on SQLite3 and isolation, I did try using two separate distinct db connections, one for ElectricSQL and one for transactions. This did not change the results.
Dumped the pragma values of the connections and didn't see anything of interest? connections-pragma.json
The test framework is Jepsen. A docker environment and GitHub action have been created that reliably reproduce the behavior. The GitHub action is a good place to start, https://github.com/nurturenature/jepsen-causal-consistency/actions
The test creates two artifacts, a result summary and the full test results.
In the results, look for
:G1b
anomalies with:reads-of-final
values. These are specifically intermediate reads.The full results:
have complete transaction, node, sync service, postgres, etc. logs.
The causal directory has further cycle and graph analysis. Intermediate reads will be graphed as a
G-single-item
. The graph for the example given above's intermediate read cycle:I'm sure there will be questions so please ask. I also hope to make the Jepsen Docker environment as usable as possible so feedback welcome.
Thanks.