Yelp / elastalert

Easy & Flexible Alerting With ElasticSearch
https://elastalert.readthedocs.org
Apache License 2.0
7.99k stars 1.73k forks source link

Flatline alert - trigger when log source is unavailable #3192

Open rejszynski opened 3 years ago

rejszynski commented 3 years ago

Hello, I am struggling with flatline type of alert, when I want to set it to trigger an alarm when a log source is unavailable (number of events is below configured threshold). This is my rule definition:

threshold: 10
timeframe:
  hours: 8
query_key: logsource
realert:
  hours: 2
use_terms_query: true
doc_type: _doc

I am still getting alerts triggered even when I have 15 million documents per log source. This is notifcation from today:

Temat: Following alert Syslog - DNS - Log source unavailable occured at 2021-06-15T00:42:18.901090Z

| Alert rule: | Syslog - DNS - Log source unavailable
-----------------------
| Timestamp:  | 2021-06-15T00:42:18.901090Z
-----------------------
| Logsource:   | ns1-bind9

An abnormally low number of events occurred around 2021-06-15 02:42 CEST.
Between 2021-06-14 18:42 CEST and 2021-06-15 02:42 CEST, there were less than 10 events.

And in this timeframe I had over 20 million documents from this logsource. I am missing some point. Could someone tell me what is the issue here?