TimelyDataflow / differential-dataflow

An implementation of differential dataflow using timely dataflow on Rust.
MIT License
2.54k stars 183 forks source link

Holding on to a trace with physical/logical compaction to the empty frontier stalls compaction #398

Open antiguru opened 1 year ago

antiguru commented 1 year ago

(Placeholder, @antiguru to add more info+repro)

While working on https://github.com/MaterializeInc/materialize/pull/18766, we noticed that allowing physical and logical compaction of a trace to the empty frontier prevents compaction, even if other owners of the trace eventually allow compaction. It is required to repeatedly allow compaction with a sensible frontier (i.e., read_upper followed by allowing compaction) to enable compaction of the spine.