search

elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
12.08k stars 4.89k forks source link

[metricbeat] aws cloudwatch metric collection optimizations #33195

Closed tommyers-elastic closed 4 months ago

tommyers-elastic commented 1 year ago

When testing cloudwatch metric collection for EC2 instances in our preprod environment, which has of the order of 10s of instances, we see metric collections taking longer than the default 5m collection period. This leads to fragmented collections and dashboards which look like they are missing data.

Investigations into the reasons why are ongoing within the cloud monitoring team. But the expected outcome is some optimizations to improve the performance of the metric collection code for cloudwatch. This is inextricably linked to https://github.com/elastic/integrations/issues/2913, but we are approaching this from the performance side only, not cost. If issues end up improving both issues, that's great.

botelastic[bot] commented 10 months ago

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!