Closed xitep closed 7 years ago
When not caching state instances, it appears worth while to avoid frequent translation of euphoria storage descriptors into flink state descriptors. As part of the initialization of the state descriptors and the state storage objects, flink's type inference and serializer initialization is at work, adding constant overhead per element/window/state to a euphoria flow, which on an equivalent flink program is present only once at the start of the program.
I've pushed some experiments into the brach pete/experimental-state-descriptors.
This was resolved by #69
AbstractWindowOperator which is part of the Euphoria's RSBK implementation for the streaming flink executor caches once retrieved states per key/window in memory. While this seemed a good idea initially, it turns out to be problematic when a lot of different keys with lot of different windows are open - this is easily achievable with time-sliding for example. Further, it turns out that caching the state objects doesn't bring much!