search

cockroachdb / pebble

RocksDB/LevelDB inspired key-value database in Go
BSD 3-Clause "New" or "Revised" License
4.76k stars 440 forks source link

db: weaken SINGLEDEL semantics in Iterator #3151

Open sumeerbhola opened 8 months ago

sumeerbhola commented 8 months ago

pebble.Iterator treats SINGLEDEL the same as DEL, under the assumption that SINGLEDEL has been correctly used. However, if used incorrectly (e.g. SINGLEDEL layered above multiple older SETs), and before the incorrect behavior has been compacted away (either harmlessly or harmfully), user-facing iteration over the data will hide the incorrectness. If we treat SINGLEDEL as deleting only a single SET, then the incorrectness will be visible -- this may be better in tests, and in replica divergence detection.

Relates to https://github.com/cockroachdb/cockroach/pull/116218 https://github.com/cockroachdb/cockroach/issues/114421

@jbowens

Jira issue: PEBBLE-122

jbowens commented 8 months ago

There's a complication here with the obsolete bit, right? Do you have any ideas on how to work around that?

Edit: Oh, if a single delete removes a key within the same sstable, we could treat both as obsolete?