integritee-network / worker

Integritee off-chain worker and sidechain validateer
Apache License 2.0
90 stars 47 forks source link

Integration test for sidechain finality #1063

Open murerfel opened 2 years ago

murerfel commented 2 years ago

In order to test our sidechain fork mitigation (i.e. proper handling of sidechain finality), we should implement an integration test that runs on CI that handles a fork situation between workers.

This test can follow the same pattern as our existing integration tests, meaning a docker compose setup for a single node and two workers. In addition, though, we need a tool like pumba to produce a fork on our sidechain. A docker compose setup with pumba was implemented in PR #744. It introduces a significant network delay on one of the worker, resulting in a fork. Based on this, we can write a test script that verifies that the worker can recover from the fork and that the sidechain state is consistent again.

murerfel commented 2 years ago

In a CI run https://github.com/integritee-network/worker/actions/runs/3383510502 I believe I've found an actual fork happening in the integration test. I'll append the corresponding logs here (fork seems to happen at around sidechain block 300/301). This can be used to compare to potential forks in the future.

logs-M6-sidechain-fork.zip