Yelp / elastalert

Easy & Flexible Alerting With ElasticSearch
https://elastalert.readthedocs.org
Apache License 2.0
8k stars 1.73k forks source link

Combining two or more rule files into a single rule file #1026

Open AviralSri opened 7 years ago

AviralSri commented 7 years ago

Hi, I have created 2 separate rule files , which are as follows:- //-------------------------------------------------------- realert:   minutes: 5 from_addr: test@email.com es_host: xx.xx.xxx.xx index: topbeat-* smtp_host: ismtp.corp.company.com type: frequency es_port: 9200 filter:  - range:     mem.used_p:       from: 0.70       to: 1.0  - term:      beat.hostname: xxxxx timeframe:   minutes: 30 alert: email name: 9serverxxxxx__mem.used_p0.7030 email: ["user@email.com"] num_events: 1 //------------------------------------------------------------

//------------------------------------------------------------ realert:   minutes: 5 from_addr: test@email.com es_host: xx.xx.xxx.xx index: packetbeat-* smtp_host: ismtp.corp.company.com type: frequency es_port: 9200 filter:  - term:      http.code: 404  - term:      beat.hostname: yyyyy timeframe:   minutes: 30 alert: email name: 25appyyyyyhttp.code404130 email: ["user@email.com"] num_events: 1

//------------------------------------------------------------

Both rule files are generating emails as per their definition.

Is there any way to have these two rule files as a single rule file. where I might need to define, index:topbeat-,packetbeat- Then in that case how I need to write filters, so that mem.used_p is queried against topbeat- for server xxxxx and http.code is queried against packetbeat- for server yyyyy. ???

AviralSri commented 7 years ago

Hi, Can anybody please reply for the above issue. Is it possible in elastalert that we query topbeat- index for some condition1 and packetbeat- index for some other condition2, and generate mail only if both condtion are satisfied. If yes, then what will be the syntax for that approach ??

Qmando commented 7 years ago

I think this can be achieved using cardinality.

type: cardinality
index: "packetbeat-*,topbeat-*"
filter:
 - query:
      query_string:
        query: "(http.code: 404 AND beat.hostname: yyyyy) OR (mem.used_p:[.7 TO 1] AND beat.hostname: xxxxx)"
cardinality_field: "beat.hostname"
max_cardinality: 1
timeframe:
  minutes: 10

Basically what this will do is get documents that match either alert. Since there's two possible hostnames, we know that both match when the cardinality of beat.hostname is greater than 1. Both of these events must be within the timeframe from each other.

There's a couple other ways to achieve this, so let me know if this works for you.