facebook / rocksdb

A library that provides an embeddable, persistent key-value store for fast storage.
http://rocksdb.org
GNU General Public License v2.0
27.83k stars 6.2k forks source link

Calculate `injected_error_count` even when `SharedState::ignore_read_error` #12800

Closed hx235 closed 1 week ago

hx235 commented 1 week ago

Context/Summary:

injected_error_count is needed to verify read error injection. For example, when injected_error_count == 0, the read call should not return error. https://github.com/facebook/rocksdb/commit/981fd432fa2441fc10a59a462bd14906ccb1c0e0 only calculated injected_error_count under SharedState::ignore_read_error=false so injected_error_count==0 when SharedState::ignore_read_error=true. However we can still inject read error in critical read path under SharedState::ignore_read_error=true so the read call is expected to return injected error. This contradicts to the injected_error_count == 0 as we skipped its calculation. As a consequence, we see

TestPrefixScan error: IO error: injected read error; 
Verification failed

in code paths

if (s.ok()) {
    thread->stats.AddPrefixes(1, count);
  } else if (injected_error_count > 0 && IsRetryableInjectedError(s)) {
    fprintf(stdout, "TestPrefixScan error: %s\n", s.ToString().c_str());
  } else {
    fprintf(stderr, "TestPrefixScan error: %s\n", s.ToString().c_str());
    thread->shared->SetVerificationFailure();
}

Test: CI

facebook-github-bot commented 1 week ago

@hx235 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 week ago

@hx235 merged this pull request in facebook/rocksdb@e90e9153d5169f9df8ddd3cf0bf693572768a7b5.