TimelyDataflow / differential-dataflow

An implementation of differential dataflow using timely dataflow on Rust.
MIT License
2.54k stars 183 forks source link

Generalize `TraceFrontier` with `upper` frontier #370

Closed frankmcsherry closed 2 years ago

frankmcsherry commented 2 years ago

This PR adds an upper: Antichain<T> frontier to TraceFrontier, generalizing it from the time interval [since, ...) to [since, upper). One option for upper is the empty frontier, which is semantically identical. This generalization allows one to import and use an arrangement on an intentionally narrowed interval of time, for example such as when one knows that changes at times greater than or equal to upper are of no interest.

This PR causes the import operator to drop its capabilities at this point, so that it may shut down the dataflow. This does risk miscommunicating that the trace is now "complete", but only in the same sense that pressing the returned ShutdownButton does, so this is judged to be not a new risk.

frankmcsherry commented 2 years ago

Thanks for the review!