We could actually support concurrent merges and tombstone cleaning if we know the partitioning ahead of time (e.g. month), and can put "bounds" on the logic to say that parts can be merged [lower bound, upper bound) (exclusive to prevent conflicts).
These must be optional.
Then locks could acquire their part ranges, allowing for n concurrent merges since there would be no conflicts.
This could provide a HUGE performance boost for merging, and thus making queries on recent data much faster, esp a higher insert rates.
We could actually support concurrent merges and tombstone cleaning if we know the partitioning ahead of time (e.g. month), and can put "bounds" on the logic to say that parts can be merged
[lower bound, upper bound)(exclusive to prevent conflicts).These must be optional.
Then locks could acquire their part ranges, allowing for
nconcurrent merges since there would be no conflicts.This could provide a HUGE performance boost for merging, and thus making queries on recent data much faster, esp a higher insert rates.