TimelyDataflow / timely-dataflow

A modular implementation of timely dataflow in Rust
MIT License
3.25k stars 273 forks source link

input handles: give empty containers to operators #525

Closed petrosagg closed 1 year ago

petrosagg commented 1 year ago

The current code tried to be helpful by ignoring any messages with zero length, but that made it impossible for the caller to differentiate between "There was an empty container message" vs "There are no more messages".

In the presence of empty containers in the channels this can lead operators to prematurely yield back to timely without fully processing their inputs, which in turn can lead to the whole dataflow not making progress since timely won't re-activate operators that have leftover events in their channels.

This PR removes this check and always forwards the containers to the operators, exactly as they were sent.

Fixes #523