This omission seems to be because these are ECS fields. While this is acceptable if the service has metadata, such as compute, it results in missing crucial labels for services without additional metadata logic.
For instance, when fetching the instance/leader_percentage_by_region metric for the spanner service, the instance_id label is missing. This leads to a discrepancy between what we deliver and what users see in the GCP Metrics Explorer.
We need to ensure these ECS fields are included even if the service lacks metadata.
When parsing time series objects, we're omitting key labels like
instance_id,zone, andproject_id.https://github.com/elastic/beats/blob/c2b2983c1825dcf7dabf64c4a04ce9cf34814fce/x-pack/metricbeat/module/gcp/timeseries_metadata_collector.go#L103
This omission seems to be because these are ECS fields. While this is acceptable if the service has metadata, such as
compute, it results in missing crucial labels for services without additional metadata logic.For instance, when fetching the
instance/leader_percentage_by_regionmetric for thespannerservice, theinstance_idlabel is missing. This leads to a discrepancy between what we deliver and what users see in the GCP Metrics Explorer.We need to ensure these ECS fields are included even if the service lacks metadata.
cc: @ishleenk17