common-streams has a feature in which a timed window of events is allowed to start processing before the previous window has finalized. This is a great feature for making full use of available cpu. It means are are always working the cpu hard, even if some slow I/O is required to finalize the window.
Until now, the eagerness only stretched to consecutive windows. E.g. If window 1 is still finalizing then it is allowed for window 2 to start processing; but it is not allowed for window 3 to start processing.
For Lake Loader I found it is better to let the eagerness stretch further. E.g. window 3 is allowed to start processing even if windows 1 and 2 are both still finalizing.
This PR makes configurable how many windows may start eagerly ahead of a finalzing window.
common-streams has a feature in which a timed window of events is allowed to start processing before the previous window has finalized. This is a great feature for making full use of available cpu. It means are are always working the cpu hard, even if some slow I/O is required to finalize the window.
Until now, the eagerness only stretched to consecutive windows. E.g. If window 1 is still finalizing then it is allowed for window 2 to start processing; but it is not allowed for window 3 to start processing.
For Lake Loader I found it is better to let the eagerness stretch further. E.g. window 3 is allowed to start processing even if windows 1 and 2 are both still finalizing.
This PR makes configurable how many windows may start eagerly ahead of a finalzing window.