search

TU-Berlin-DIMA / scotty-window-processor

This repository provides Scotty, a framework for efficient window aggregations for out-of-order Stream Processing.
https://tu-berlin-dima.github.io/scotty-window-processor/
Apache License 2.0
75 stars 23 forks source link

LazyEvaluation of the "Lift" function. #13

Open tuterbatuhan opened 5 years ago

tuterbatuhan commented 5 years ago

Hey, "Lift" function is lazily evaluated by calling the getAggValues() function, as it can be seen here: https://github.com/TU-Berlin-DIMA/scotty-window-processor/blob/a18a7dc101f814b41a416aca9785a5848221667d/flink-connector/src/main/java/de/tub/dima/scotty/flinkconnector/demo/FlinkQuantileDemo.java#L28-L32

You may consider triggering the "Lift" function when a window ends, instead of adding another operator for lazy evaluation.

PhilippGrulich commented 5 years ago

Hi @tuterbatuhan,

we will consider this for future versions of Scotty.

Best, Philipp

jiayuasu commented 2 years ago

Hi all, I wonder why this getAggValues() is lazily evaluated? Due to some performance concern?

julianev commented 2 years ago

Hi @jiayuasu, The lazy evaluation, in this case, is related to the quantile computation. We do not expect a great performance benefit from changing it. If you have further questions, please feel free to contact us!

Best, Juliane