Closed sfc-gh-jslocum closed 2 years ago
The test is failing with the following event sequence:
failed
movekeys_conflict
which will trigger the remove server process, however, this never happened. When the SS1 tracker checks the excludedServers after a while, it didn't see the FAIL
status, the status was reset somehow, I traced all invocations of excludedServers.set()
, but was not able to find out who reset the status.It turns out the remote TeamCollection never started since the remote DC is not fully recovered. I will disable HA mode for the test since the feature was never meant for HA anyway.
The fix was checked in and test re-enabled. https://github.com/apple/foundationdb/pull/5868
DataLoss Recovery has been failing regularly in the snowflake nightlies. Until the issues with this test can be fixed, it was disabled in https://github.com/apple/foundationdb/issues/5847 (PR https://github.com/apple/foundationdb/pull/5848).
Once the issues are fixed, and DataLossRecovery passes a sufficient number of joshua runs with no failures, we should re-enable the test on master.