search

Kristina-Pianykh / flink-multinode

Other
0 stars 0 forks source link

[feat]: set static flag to inject into incoming Events to control which pattern to apply #3

Closed Kristina-Pianykh closed 3 weeks ago

Kristina-Pianykh commented 2 months ago

The Flink lib unfortunately still doesn't offer the feature of dynamically updating the CEP patterns at runtime (to add or remove them based on application changes). But there's a workaround.

  1. Create a static flag (e.g. multiSinkEnabled) to reflect whether the multi-sink pattern is enabled. Where: the socket source function.
  2. Inject the flag value into the event created from the socket stream
  3. update the multi-sink pattern object to match based on the flag value of the event

This will affectively not apply the multi-sink pattern anymore after the switch which is equivalent to removing it altogether.

Kristina-Pianykh commented 2 months ago

The events are checked in the socket source function for control messages: https://github.com/Kristina-Pianykh/flink-multimetric-alert/blob/b629f79b1c0ecb42946a2ff825f671dd2f3ea9aa/src/main/java/flink_state_access/SocketSource.java#L55-L59

The boolean flag to enable/disable the target pattern is globally scoped within the socket source function: https://github.com/Kristina-Pianykh/flink-multimetric-alert/blob/b629f79b1c0ecb42946a2ff825f671dd2f3ea9aa/src/main/java/flink_state_access/SocketSource.java#L9

The pattern includes the flag check: https://github.com/Kristina-Pianykh/flink-multimetric-alert/blob/b629f79b1c0ecb42946a2ff825f671dd2f3ea9aa/src/main/java/flink_state_access/StreamingJob.java#L40