search

elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
96 stars 4.92k forks source link

[Metricbeat] Investigate collection period=1m for googlecloud module #17141

Closed kaiyan-sheng closed 4 years ago

kaiyan-sheng commented 4 years ago

By looking at monitoring metrics on google cloud platform console, they ones I saw all have 1 minute interval. For example:

Screen Shot 2020-03-19 at 2 44 11 PM

Also with current code, when period = 5min is set, metrics are collected per minute for 2 or 3 out of the 5 minutes. I would like to see either collecting every minute in each 5-minute instead of partially. Please see kibana snapshot for this:

Screen Shot 2020-03-09 at 1 12 13 PM
elasticmachine commented 4 years ago

Pinging @elastic/integrations-platforms (Team:Platforms)

kaiyan-sheng commented 4 years ago

GCP monitoring metrics latency: Sampled every 60 seconds. After sampling, data is not visible for up to 240 seconds.

exekias commented 4 years ago

We had a chat about this and I think we need to change some things. It sounds like we should collect more often but account for the 240 seconds delay when choosing the time range. Also find a way to avoid sending duplicate data