search

afedulov / fraud-detection-demo

Repository for Advanced Flink Application Patterns series
https://flink.apache.org/news/2020/01/15/demo-fraud-detection.html
331 stars 150 forks source link

Add Rule validation to check if all fields in rule are present or not. #27

Open galadrielwithlaptop opened 1 year ago

galadrielwithlaptop commented 1 year ago

First, I want to convey that it is a great demo to showcase the power of Flink. As I am running this on my environment, if I add a rule whose windowMinutes is not defined, flink throws an exception.

java.lang.NullPointerException     at com.ververica.field.dynamicrules.Rule.getWindowMillis(Rule.java:46)     at com.ververica.field.dynamicrules.functions.DynamicAlertFunction.updateWidestWindowRule(DynamicAlertFunction.java:220)     at com.ververica.field.dynamicrules.functions.DynamicAlertFunction.processBroadcastElement(DynamicAlertFunction.java:144)     at com.ververica.field.dynamicrules.functions.DynamicAlertFunction.processBroadcastElement(DynamicAlertFunction.java:51)     at org.apache.flink.streaming.api.operators.co.CoBroadcastWithKeyedOperator.processElement2(CoBroadcastWithKeyedOperator.java:133)     at org.apache.flink.streaming.runtime.io.StreamTwoInputProcessorFactory.processRecord2(StreamTwoInputProcessorFactory.java:225)     at org.apache.flink.streaming.runtime.io.StreamTwoInputProcessorFactory.lambda$create$1(StreamTwoInputProcessorFactory.java:194)     at org.apache.flink.streaming.runtime.io.StreamTwoInputProcessorFactory$StreamTaskNetworkOutput.emitRecord(StreamTwoInputProcessorFactory.java:266)     at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)     at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)     at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)     at org.apache.flink.streaming.runtime.io.StreamMultipleInputProcessor.processInput(StreamMultipleInputProcessor.java:85)     at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:542)     at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)     at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)     at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)     at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)     at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)     at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)     at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)     at java.base/java.lang.Thread.run(Unknown Source)

After analysis, if a new rule is added where windowMinutes is not defined, it throws an error.

I can take this up and add validation check for each new rule. :)

Thanks in advance!!

afedulov commented 1 year ago

Hi @galadrielwithlaptop , sure, feel free to open a PR.