db: allow configuring depth of manual compactions

cockroachdb / pebble

RocksDB/LevelDB inspired key-value database in Go

BSD 3-Clause "New" or "Revised" License

4.76k stars 441 forks source link

db: allow configuring depth of manual compactions #1622

Open jbowens opened 2 years ago

jbowens commented 2 years ago

Currently DB.Compact compacts the entirety of the provided key range, ensuring all the data from the mutable memtable through L6 is compacted into L6. In cockroachlabs/support#1516 we used compact_engine_span. For our purposes, we could've compacted just the span from L0->L1.

Jira issue: PEBBLE-101

bananabrick commented 2 years ago

@jbowens How large was L0, and how long did that online manual compaction take?

jbowens commented 2 years ago

I don't know. There many nodes with accumulation of many-file single-sublevel L0s, and we issued the manual compaction across many of them. We didn't both monitoring the online compaction after it was triggered.

github-actions[bot] commented 5 months ago

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to Pebble!