Elastalert aggregate all events into one alert, notify once per day

[akshays958]

Hello, I am new to elastalert and want my rule to execute and accumulate alerts for one day, and at the end, notify me about all the alerts through the day in one single alert. I do not want descriptive messages in my alert, just the number of hits. I am using Slack channels for alerting.

This is how far I have gotten:

es_host: ###
es_port: ###
es_conn_timeout: 200

name: test-frequency-rule
type: frequency
index: ###
use_count_query: true
doc_type: ###
run_every:
  days: 1
buffer_time:
  days: 1
num_events: 1
timeframe:
  days: 1
aggregation:
  days: 1
realert:
  days: 1
filter:
- query:
    query_string:
        query: "########"
alert:
- "slack"
include: ["_index", "host", "num_hits", "_type", "logLevel"]
slack:
slack_webhook_url: ###

[Qmando]

You want the total number of matching documents in a single line, every day?

If you are using 1 day run_every, you can remove aggregation. Also, you can remove buffer_time because it doesn't do anything when use_count_query is used. You can also remove include because it's not used with use_count_query. I'd also remove realert, since it should only be running once per day anyway, though this probably doesn't make a difference.

You probably want to add

alert_text_type: alert_text_only
alert_text: "Number of hits: {}"
alert_text_args: [num_hits]

If you want no descriptive text.

[akshays958]

Thanks for your quick reply. Also I have another rule that I need to run so that it gets alerts after, suppose 6 hrs, with only one alert, containing all events. I do not want alerts within 6 hrs, just at the end of 6 hrs. I tried doing the following but it seems to give alert at every hit/match:

es_host: ######
es_port: ####
es_conn_timeout: 200

name: ######
type: any
index: ###

run_every:
  hours: 6
buffer_time:
  hours: 6
num_events: 1
timeframe:
  hours: 6
aggregation:
  hours: 6
filter:
- query:
    query_string:
      query: '#####'
alert:
- "slack"
include: ["_index", "host", "message", "num_hits", "_type"]
slack:
slack_webhook_url: "######"

[Qmando]

You have aggregation set but you get alerts immediately? That doesn't seem right to me. By the way, you can remove buffer_time, run_every, num_events, and timeframe from this rule and it should do exactly what you want.

[szabogergo]

Maybe I have a same problem with sending alerts. I would like to make an alert rule which helps me to sending one aggregated reporting alert per day and within alert text a want to display network devices' hostname, the unique log entry related to them and also the cardinality of each log entry.

I started using "any" rule with aggregation options:

type: any
aggregation:
  days: 1
  schedule: '0 5 * * * *'
aggregation_key: 'hostname'
summary_table_fields:
  - hostname
  - log_entry
alert_text_type: aggregation_summary_only

With this solution I get the following data in one row: _hostname - log_entry - log_entrycount and unfortunatelly, I get as many e-mails separately as many hosts send some log_entry.

The goal should be something similar in one alert per day:

hostname1 - log_entry1 (log_entry1_count), log_entry2 (log_entry2_count), ... 
hostname2 - log_entry1 (log_entry1_count), log_entry2 (log_entry2_count), ...
 ... 
hostnameN - log_entry1 (log_entry1_count), log_entry2 (log_entry2_count), ...

Is it possible somehow?