Yelp / elastalert

Easy & Flexible Alerting With ElasticSearch
https://elastalert.readthedocs.org
Apache License 2.0
7.99k stars 1.74k forks source link

Elast Alert CPU usage/Memory usage reporting values are incorrect. #3110

Open saiprathapdp opened 3 years ago

saiprathapdp commented 3 years ago

Hi,

We have configured Elastalert to alert cpu/memory/ utilization is used more than 80 percent. And we are also receiving the email alert notification, elastalert is not triggering the correct values in the alerting.

We have manually checked in that customer host itself the CPU/Memory utilization is 30 percent only but the elastalert showing 80 percent. Find the below Elastalert version and configuration file.

image

Memory Warning rule:

es_host: node1.elkstack.es.local

es_port: 9200

name: metricbeat_Warning_memory_rule

type: any

index: metricbeat*

metric_agg_key: system.memory.actual.used.pct metric_agg_type: avg query_key: host.hostname doc_type: _doc

sync_bucket_interval: true buffer_time: minutes: 1 bucket_interval: minutes: 1

realert: minutes: 60

filter:

Postfix-SMTP Details

smtp_host: mail.rbeicloud.net smtp_port: 25 from_addr: elkadmin@mail.rbeicloud.net

alert_subject: "{3}-IaaS-Memory-Warning - encountered threshold for {2} host: {0} at {1}%" alert_subject_args:

alert_text_type: alert_text_only alert_text: "Elasticsearch Index: {0}\n\nCloud_Details:\nInstance_Id: {1}\nMachine_Type: {2}\nProvider: {3}\nRegion: {4}\n\nHost_Details:\nArchitecture: {5}\nHostname: {6}\nFamily: {7}\nName: {8}\nPlatform: {9}\nVersion: {10}\n\nMemory Used (0/10): {11}%" alert_text_args:

Alert Type

alert:

CPU Warning rule:-

es_host: node1.elkstack.es.local

es_port: 9200

name: metricbeat_Warning_cpu_rule

type: any

index: metricbeat*

metric_agg_key: metric_agg_script metric_agg_script: script: (doc['system.cpu.user.pct'].value + doc['system.cpu.system.pct'].value) / doc['system.cpu.cores'].value metric_agg_type: avg query_key: host.hostname doc_type: _doc

sync_bucket_interval: true buffer_time: minutes: 1 bucket_interval: minutes: 1

realert: minutes: 60

filter:

Postfix-SMTP Details

smtp_host: mail.rbeicloud.net smtp_port: 25 from_addr: elkadmin@mail.rbeicloud.net

alert_subject: "{3}-IaaS-CPU-Warning - encountered threshold for {2} host: {0} at {1}%" alert_subject_args:

alert_text_type: alert_text_only alert_text: "Elasticsearch Index: {0}\n\nCloud_Details:\nInstance_Id: {1}\nMachine_Type: {2}\nProvider: {3}\nRegion: {4}\n\nHost_Details:\nArchitecture: {5}\nHostname: {6}\nFamily: {7}\nName: {8}\nPlatform: {9}\nVersion: {10}\n\nCPU Used (0/10): {11}%" alert_text_args:

Alert Type

alert:

Please let us know the metricbeat agent version has to be update or any other configuration needs to be change.