Suppress functionality

[agayev169]

I was looking for the suppression over table functionality as in the Java version of Kafka Streams. Is it planned to be added in the future?

[LGouellec]

You seems mentionned this feature ? :

KTable<String, String> table = builder.table("topic");
table.suppress(...);

It's planned on my roadmap, not in future release 1.2.0, but surely in 1.3.0 release.

[agayev169]

@LGouellec do you think it is possible to add this functionality in the next release if it is implemented till that time?

[LGouellec]

@agayev169 to finish next release, I miss this PR. Persistent Preview State Store with RocksDb State store implementation (key/value, timestamp & window). I hope release this version end April / begin May. Unfortunately, I can't have enought time to implement Supress(..) processor. So if you want contribute, please feel free to create a PR. It could cool :)

[agayev169]

Great! I will have a look at this problem, and if I am able to finish it I will create a PR. Thank you for the fast response!

[yornstei]

@agayev169 @LGouellec is there any progress/plan to add this functionality?

[LGouellec]

Hi @yornstei ,

I updated the compare list, https://github.com/LGouellec/kafka-streams-dotnet/tree/1.4#compare-kafka-streams-vs-streamiz

For now, the suppress(..) processor is not present in any roadmap.

Do you have any workaround regarding this feature ?

Best regards,

[yornstei]

@LGouellec thanks for the reply.

I'd like to emit an event if within a specific window a status is updated more than x amount of times for a particular key, and only get notified at the end of the window. It seems like the suppress functionality is build for this and I cannot think of an elegant way to accomplish this without it.

Any idea how to go about this without suppress?

Thanks for your help!

[LGouellec]

@yornstei

Unfortunately no for now. You have the transform API will be release in 1.4.0 but you haven't the possibility to downstream record for now.

Best regards,

[LGouellec]

I will reopen this issue and track the up-vote for 1.5.0 or 1.6.0

[oroiandan193]

Hi @LGouellec , Any idea/workaround to suppress event bursts which go from mongo to ES through kafka connector ? I am also listening to that topic to react to entity changes but they are too many. Was thinking about groupby, aggregate and take last event in a windowed time which is suppressed.

How`s roadmap looking so far ? :) when can we expect 1.7.0 ? Thx, much appreciated !

[LGouellec]

@oroiandan193, No today without this suppress feature, you can't remove the intermediate records. It's flagged for 1.7.0 because before that, the library must a cache feature flagged for 1.6.0

[tvergilio]

Is there a workaround for this? I would like to wait until all the events for a window have arrived, then perform some aggregation and emit one result, once, at the watermark (end of the window + configured lateness tolerance).

[LGouellec]

Hi @tvergilio ,

Today, the only workaround is to implement your own processors but it's not so easy ! Supress(..) is part of the 1.7.0 milestone, which will be release Q4 2024.

[tvergilio]

I agree, it is not straightforward. Looking forward to the next release, this is a powerful feature which could enable more advanced aggregation and stream processing abstractions. Thanks for the quick reply.

[LGouellec]

PR #377 Merged in develop Plan for release 1.7.0 Q4 2024