Due to a few data race issues, sometimes active WAL file gets copied in inconsistent state during checkpoint operation.
Database open from such checkpoint fails with one of these errors when wal_recovery_mode=kAbsoluteConsistency:
Corruption: truncated record body
Corruption: error reading trailing data
This happens because size of the active WAL file is captured at a random moment:
truncated record body error happens when WAL file size is captured right after WritableFileWriter flush when in-memory buffer no longer has space for new data
error reading trailing data happens, when WAL record gets broken down into multiple physical records(fragments), and WAL file size was captured before last fragment has been written.
The fix does the following:
keeps track of the latest offset where WAL file is consistent in log::Writer;
in GetLiveFilesStorageInfo, captures current log number (if there is active WAL) and it's latest consistent offset, right after capturing manifest and options state;
after enumerating WAL files in directory, for active WAL, sets trim_to_size=true + size to the consistent offset, captured previously;
WAL files, created during GetLiveFilesStorageInfo call (between mutex_.Unlock and GetSortedWalFiles) are ignored, since we don't know which portion of such WAL we have to copy so it would be copied in a consistent state.
Due to a few data race issues, sometimes active WAL file gets copied in inconsistent state during checkpoint operation. Database open from such checkpoint fails with one of these errors when
wal_recovery_mode=kAbsoluteConsistency:Corruption: truncated record bodyCorruption: error reading trailing dataThis happens because size of the active WAL file is captured at a random moment:
truncated record bodyerror happens when WAL file size is captured right afterWritableFileWriterflush when in-memory buffer no longer has space for new dataerror reading trailing datahappens, when WAL record gets broken down into multiple physical records(fragments), and WAL file size was captured before last fragment has been written.The fix does the following:
log::Writer;GetLiveFilesStorageInfo, captures current log number (if there is active WAL) and it's latest consistent offset, right after capturing manifest and options state;trim_to_size=true+ size to the consistent offset, captured previously;GetLiveFilesStorageInfocall (betweenmutex_.UnlockandGetSortedWalFiles) are ignored, since we don't know which portion of such WAL we have to copy so it would be copied in a consistent state.Fixes #12670