The following rule works and will send me an alert if the combination of host_name and log.file.path has more than 500 matches.
name: Logstash failure to send to Kafka
type: frequency
num_events: 500
index: logstash-logs*
realert:
hours: 1
timeframe:
minutes: 15
doc_type: logstash-log
query_key:
- host_name
- log.file.path
filter:
- term:
logEvent.message: Sending batch to Kafka failed. Will retry after a delay.
# only need the hostname and log.file.path fields for my alert
alert:
- slack
The problem is that this query may match a very large number of documents and I've seen this put significant memory pressure on the container I'm running Elastalert in. I have tried to change the rule to the following use_terms_query rule
name: Logstash Send to Kafka Failed
type: frequency
num_events: 500
index: logstash-logs*
realert:
hours: 1
timeframe:
minutes: 15
use_terms_query: true
doc_type: logstash-log
terms_size: 400
query_key:
- host_name
- log.file.path
filter:
- term:
logEvent.message: Sending batch to Kafka failed. Will retry after a delay.
# only need the hostname and log.file.path fields for my alert
alert:
- slack
However, this rule is not working. I grabbed the es_query from the trace file by running elastalert with the --es_debug_trace option and I can see that the field for the terms aggregation is just a comma delimited list from my query_key
The use_terms_query works for me if I only specify one term in the query_key (like host_name), however I'd like to alert on the combination of fields without having to resort to concatenating the fields into a single composite field for aggregation purposes. Is the lack of support for multiple query_keys in the use_term_query a bug or intended? Is there are work around to my issue without having to download all the matching documents like in the first query?
The following rule works and will send me an alert if the combination of host_name and log.file.path has more than 500 matches.
The problem is that this query may match a very large number of documents and I've seen this put significant memory pressure on the container I'm running Elastalert in. I have tried to change the rule to the following use_terms_query rule
However, this rule is not working. I grabbed the es_query from the trace file by running elastalert with the --es_debug_trace option and I can see that the field for the terms aggregation is just a comma delimited list from my query_key
The field "host_name,log.file.path" does not exist. I would have expected/hoped that the the following nested aggregation would have been created.
The use_terms_query works for me if I only specify one term in the query_key (like host_name), however I'd like to alert on the combination of fields without having to resort to concatenating the fields into a single composite field for aggregation purposes. Is the lack of support for multiple query_keys in the use_term_query a bug or intended? Is there are work around to my issue without having to download all the matching documents like in the first query?