search

TimelyDataflow / timely-dataflow

A modular implementation of timely dataflow in Rust
MIT License
3.25k stars 272 forks source link

dataflow: Account for probe handles being dropped #490

Open petrosagg opened 1 year ago

petrosagg commented 1 year ago

The lifetime of a probe handle is not attached to any particular dataflow and users are free to connect them to streams of multiple dataflows.

This PR handles the situation in which a probe handle is connected to two separate dataflows and one of them drops. Previously the handle would have its frontier stuck because the old handle wouldn't provide any updates anymore.

The implemented solution is that each handle keeps track of its own changes and removes them from the overall calculation on drop.