[GCP] Optimize `metrics` metricset and metadata collection

[gpop63]

Overview

We've observed performance issues with the GCP metrics metricset when collecting data from a large number of projects with many instances. This affects services with or without additional metadata collection.

Current behavior:

• No caching mechanism
• Metadata collected every period (usually 1 minute)
• All fields from GCP responses are collected, potentially including unused data

Google's official recommendations for optimizing API calls (https://cloud.google.com/resource-manager/docs/performance) suggest:

• Compression using gzip
• Partial responses:
  • Selecting specific fields needed from the response
  • Note: This requires using the older library (google.golang.org/api/compute/v1) as the newer one (cloud.google.com/go/compute/apiv1) doesn't support this feature yet

Test Results (instances.aggregatedList)

Tested on my machine, so results may vary on other systems.

Using custom fields significantly reduced response size (from ~24 MB to 0.146 MB) and improved performance (from 4.05s to 2.10s). The impact of gzip is not so noticeable in this test maybe because it's small scale, but it could be more significant with larger datasets or over slower network connections.

Custom fields used in test: id,items/zones/instances(name,labels,machineType,status)

But as I mentioned earlier, services without extra metadata API calls can have issues. We should optimize the ListTimeSeries and ListMetricDescriptors calls, possibly by enabling gzip compression.

[elasticmachine]

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

[botelastic[bot]]

This issue doesn't have a Team:<team> label.