Yelp / elastalert

Easy & Flexible Alerting With ElasticSearch
https://elastalert.readthedocs.org
Apache License 2.0
8k stars 1.73k forks source link

Spike Count mismatch with Kibana logs. #3304

Open ratnakumarchukkapalli opened 3 months ago

ratnakumarchukkapalli commented 3 months ago

I have implemented one alerting rule When I trigger this alert, the count in kibana is different with count mentioned in the alert. Can you please help me why there is discrepancy in kibana and in elastalert2. Is it due to indexing? Please let me know when you are free for few min, we can have call. Thank you. (9:53 to 10:03) Count is 130 (10:03 to 10:13) Count is 137 Opsgenie alert : In opsgenie, the alert triggered at 10:15 and it says around 10:13 it was 89 and preceeding that it was 109 but in kibana it was more between 10:03 to 10:13

Previous count: 109 Current count: 89 An abnormal number (89) of events occurred around 2024-08-22 10:13 UTC. Preceding that time, there were only 109 events within 0:10:00 @timestamp: 2024-08-22T10:13:05.500033Z num_hits: 8 num_matches: 1 reference_count: 109 spike_count: 89

Below is my config timeframe: minutes: 10 timestamp_field: "@timestamp" timestamp_type: "iso" use_strftime_index: true use_count_query: true spike_type: "down"
spike_height: 1.2

realert: minutes: 10

The count is not matching with kibana logs. Please help me out. I am trying since 2months but not able to crack and the documentation is confusing