elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
12.16k stars 4.91k forks source link

[GCP] Cache GCP metadata instead of recollecting every collection period #40395

Open kaiyan-sheng opened 2 months ago

kaiyan-sheng commented 2 months ago

When we are trying to run gcp.compute metricset to collect metrics from large amount of VM instances, it takes a long time to fetch metadata from all instances and we are running this fetch every collection period. This will cause gcp.compute metricset to not able to ingest metrics.

The workaround is to specify exclude_labels parameter to true to skip metadata collection:

         if !m.config.ExcludeLabels {
        if metadataService, err = NewMetadataServiceForConfig(m.config, sdc.ServiceName); err != nil {
            return nil, fmt.Errorf("error trying to create metadata service: %w", err)
        }
    }

Since metadata for each instance does not change frequently, tt will be great if we can implement a cache for the metadata and not fetching the data every collection period.

kwinstonix commented 2 months ago

Besides compute service metrics,there are other services that have metrics with resource_type gce_instance. If there is a cache that can be used cross metricset, that would be great.