search

cockroachdb / pebble

RocksDB/LevelDB inspired key-value database in Go
BSD 3-Clause "New" or "Revised" License
4.79k stars 444 forks source link

sstable: isolate and report corruption in sstables #1192

Open itsbilal opened 3 years ago

itsbilal commented 3 years ago

This is one of the bigger Pebble-side changes as part of the sstable / replica corruption project in cockroach. As part of this, sstable readers / iterators will build out a defined reporting path for any corruption errors encountered when reading an sstable. Common corruption errors are "bad magic number", "checksum mismatch", etc., and lots of instances of them are marked as dupes of https://github.com/cockroachdb/cockroach/issues/57934 .

Pebble needs to report (through a CorruptionReporter interface) the key bounds of an instance of corruption encountered. Pebble could optionally deduplicate / merge these key-ranges of corruption, or leave that up to the entity running Pebble (Cockroach).

The easy thing to do when reporting corruption key bounds is to just report the sstable's bounds from its FileMetadata as corrupt, if a corruption error is encountered when reading it. However, in cases where the corruption is in a data block, the bounds could be narrowed further by looking at adjacent non-corrupt data blocks and figure out the closest adjoining keys to the corrupt block(s) that can be read correctly.

Jira issue: PEBBLE-210

github-actions[bot] commented 1 year ago

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to Pebble!

github-actions[bot] commented 2 months ago

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to Pebble!