When we are trying to run gcp.compute metricset to collect metrics from large amount of VM instances, it takes a long time to fetch metadata from all instances and we are running this fetch every collection period. This will cause gcp.compute metricset to not able to ingest metrics.
The workaround is to specify exclude_labels parameter to true to skip metadata collection:
if !m.config.ExcludeLabels {
if metadataService, err = NewMetadataServiceForConfig(m.config, sdc.ServiceName); err != nil {
return nil, fmt.Errorf("error trying to create metadata service: %w", err)
}
}
Since metadata for each instance does not change frequently, tt will be great if we can implement a cache for the metadata and not fetching the data every collection period.
Besides compute service metrics,there are other services that have metrics with resource_type gce_instance. If there is a cache that can be used cross metricset, that would be great.
When we are trying to run
gcp.compute
metricset to collect metrics from large amount of VM instances, it takes a long time to fetch metadata from all instances and we are running this fetch every collection period. This will causegcp.compute
metricset to not able to ingest metrics.The workaround is to specify
exclude_labels
parameter totrue
to skip metadata collection:Since metadata for each instance does not change frequently, tt will be great if we can implement a cache for the metadata and not fetching the data every collection period.