Elastalert - Timestamp problems

[Invictus96]

Hello, i created a rule any.yml and the result are 3 hits and 3 matches. But in the email body there is the content of only one of the three matches. How can I change, that in one email the content of every message_field where the rule fits is shown?

Thanks for your response

[Invictus96]

ok, realalert: minutes: 0 fixed it! But now I got three mails, how can I put the content of those three into one email?

[Qmando]

add

aggregation:
  minutes: 5

which means that after the first alert, all subsequent alerts for the next 5 minutes will be joined into a single email.

[Invictus96]

Yes, I tried this yesterday. But than no email is generated. Is there something wrong with the rule?

es_host: localhost
es_port: 9200
name: LoggingLevel
type: any
index: filebeat-*

query_key: beat.hostname

filter:
 - query_string:
    query: "message:Error" 
        #OR message:Error"
        #and source:PFAD

alert:
 - "email"

email:
 - "MyEmail"

smtp_host: "192.168.146.3" 

alert_text: |
    timestamp: {0}
    message: {1}
alert_text_args: ["@timestamp", "message"]

alert_text_type: exclude_fields 

realert:
  minutes: 0

aggregation:
  minutes: 5

On what time is the aggregation based? The timestamp or the time I run the command in cmd?

Thanks

[Invictus96]

I think there might be something wrong with the field @timestamp. The logs have the log-time at 20:04:22 but that time is ignored. In Elasticsearch the timestamp by reading the logs is 13:02:08.638 When I run the Test-Rule, the local time is used. But in the E-Mail body the timestamp: {0}-field is 11:02:08.638 So where did the program get this time from? I already found out that when I increase the aggregation time the time with the next alert will be set up.

INFO:elastalert:Adding alert for LoggingLevel to aggregation(id: , aggregation_key: None), next alert at 2017-10-20 11:59:20.799000+00:00 INFO:elastalert:Adding alert for LoggingLevel to aggregation(id: , aggregation_key: None), next alert at 2017-10-20 11:59:20.799000+00:00 INFO:elastalert:Adding alert for LoggingLevel to aggregation(id: , aggregation_key: None), next alert at 2017-10-20 11:59:20.799000+00:00

But when I set the aggregation time a bit over the actual daytime: 13:57 so that the alert should be send at for example 14:00 there is no e-mail incomming.

Has someone a hint?

[Qmando]

5 minute aggregation means 5 minutes after the first match, you will get the email. You just need to wait a couple minutes. It even says WHEN it will alert in the logs, next alert at 2017-10-20 11:59:20.799000+00:00.

If you have a second timestamp field other than @timestamp, you can use that instead by setting timestamp_field: log-time. Whatever uploaded these documents to Elasticsearch for you added @timestamp. It's probably in UTC which is why you are confused.