apple / foundationdb

FoundationDB - the open source, distributed, transactional key-value store
https://apple.github.io/foundationdb/
Apache License 2.0
14.19k stars 1.29k forks source link

Fix an assertion failure when waiting for recovery [release-7.1] #11398

Closed jzhou77 closed 1 month ago

jzhou77 commented 1 month ago

cherrypick #11399

CC's checkBetterSingletons() calls getUsedIds() that asserts proxy interfaces are present. However, when a GRV/commit proxy failed, before CC starts a new recovery, the proxy's processId becomes empty, thus triggering the failure.

The fix is to cancel the caller while waiting for recovery.

To reproduce 7.1 commit 725a08a3ff clang build:

./fdbserver.6.0.15 -r simulation -f ./tests/restarting/from_5.2.0_until_6.3.0/ClientTransactionProfilingCorrectness-1.txt -s 900000399 -b on
-f ./tests/restarting/from_5.2.0_until_6.3.0/ClientTransactionProfilingCorrectness-2.txt --restarting -s 900000400 -b on

correctness: 20240514-184616-jzhou-8376a3b76500b2be

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

foundationdb-ci commented 1 month ago

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

foundationdb-ci commented 1 month ago

Result of foundationdb-pr-macos on macOS Ventura 13.x

foundationdb-ci commented 1 month ago

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

foundationdb-ci commented 1 month ago

Result of foundationdb-pr-clang on Linux CentOS 7

foundationdb-ci commented 1 month ago

Result of foundationdb-pr on Linux CentOS 7