Getting error message has label dimensions inconsistent with previously collected metrics in the same metric family

wdhnl commented 6 years ago

Executing stackdriver_exporter as:

stackdriver_exporter --google.project-id=myproject --monitoring.metrics-type-prefixes=serviceruntime.googleapis.com/api/request_count

Then running

curl localhost:9255/metrics

Gives me

An error has occurred during metrics collection:

2 error(s) occurred:

collected metric stackdriver_consumed_api_serviceruntime_googleapis_com_api_request_count label:<name:"grpc_status_code" value:"0" > label:<name:"location" value:"us-central1" > label:<name:"method" value:"google.api.servicecontrol.v1.ServiceController.Check" > label:<name:"project_id" value:"myproject" > label:<name:"protocol" value:"" > label:<name:"response_code" value:"200" > label:<name:"response_code_class" value:"2xx" > label:<name:"service" value:"servicecontrol.googleapis.com" > label:<name:"unit" value:"1" > label:<name:"version" value:"v1" > gauge: has label dimensions inconsistent with previously collected metrics in the same metric family
collected metric stackdriver_consumed_api_serviceruntime_googleapis_com_api_request_count label:<name:"grpc_status_code" value:"0" > label:<name:"location" value:"us-central1" > label:<name:"method" value:"google.api.servicecontrol.v1.ServiceController.Report" > label:<name:"project_id" value:"myproject" > label:<name:"protocol" value:"" > label:<name:"response_code" value:"200" > label:<name:"response_code_class" value:"2xx" > label:<name:"service" value:"servicecontrol.googleapis.com" > label:<name:"unit" value:"1" > label:<name:"version" value:"v1" > gauge: has label dimensions inconsistent with previously collected metrics in the same metric family

Is there any way to use stackdriver_exporter to grab those metrics?

everesio commented 6 years ago

You can check the fix (open PR) https://github.com/frodenas/stackdriver_exporter/pull/23

austinpray-mixpanel commented 1 year ago

Still getting this on 0.13.0

docker run --user $(id -u):$(id -g) -p 9255:9255 -v $HOME/.config/gcloud:/gcloud -e GOOGLE_APPLICATION_CREDENTIALS=/gcloud/application_default_credentials.json us.gcr.io/mixpanel-tools/prometheuscommunity/stackdriver-exporter:v0.13.0 --google.project-id=mixpanel-prod-1 --monitoring.metrics-interval=5m --monitoring.metrics-offset=0s --stackdriver.backoff-jitter=1s --stackdriver.http-timeout=30s --stackdriver.max-backoff=5s --stackdriver.max-retries=0 --stackdriver.retry-statuses=503 --web.listen-address=:9255 --web.telemetry-path=/metrics --monitoring.metrics-type-prefixes serviceruntime.googleapis.com

curl localhost:9255

An error has occurred while serving metrics:

7 error(s) occurred:
* [from Gatherer #2] collected metric "stackdriver_deprecated_resource_serviceruntime_googleapis_com_reserved_metric_1" { label:<name:"credential_id" value:"" > label:<name:"location" value:"asia-southeast1" > label:<name:"param1" value:"pubsub.googleapis.com" > label:<name:"param2" value:"" > label:<name:"param3" value:"v1" > label:<name:"param4" value:"google.pubsub.v1.Publisher.Publish" > label:<name:"project_id" value:"some-gcp-project" > label:<name:"quota_location" value:"asia-southeast1" > label:<name:"quota_name" value:"pubsub.googleapis.com/regionalpublisher" > label:<name:"unit" value:"1" > gauge:<value:60152 > timestamp_ms:1692304080000 } was collected before with the same name and label values
* [from Gatherer #2] collected metric "stackdriver_deprecated_resource_serviceruntime_googleapis_com_reserved_metric_1" { label:<name:"credential_id" value:"" > label:<name:"location" value:"europe-west1" > label:<name:"param1" value:"pubsub.googleapis.com" > label:<name:"param2" value:"" > label:<name:"param3" value:"v1" > label:<name:"param4" value:"google.pubsub.v1.Publisher.Publish" > label:<name:"project_id" value:"some-gcp-project" > label:<name:"quota_location" value:"europe-west1" > label:<name:"quota_name" value:"pubsub.googleapis.com/regionalpublisher" > label:<name:"unit" value:"1" > gauge:<value:164495 > timestamp_ms:1692304080000 } was collected before with the same name and label values
* [from Gatherer #2] collected metric "stackdriver_deprecated_resource_serviceruntime_googleapis_com_reserved_metric_1" { label:<name:"credential_id" value:"" > label:<name:"location" value:"us-central1" > label:<name:"param1" value:"pubsub.googleapis.com" > label:<name:"param2" value:"" > label:<name:"param3" value:"v1" > label:<name:"param4" value:"google.pubsub.v1.Publisher.Publish" > label:<name:"project_id" value:"some-gcp-project" > label:<name:"quota_location" value:"us-central1" > label:<name:"quota_name" value:"pubsub.googleapis.com/regionalpublisher" > label:<name:"unit" value:"1" > gauge:<value:8.165689e+06 > timestamp_ms:1692304080000 } was collected before with the same name and label values
* [from Gatherer #2] collected metric "stackdriver_deprecated_resource_serviceruntime_googleapis_com_reserved_metric_1" { label:<name:"credential_id" value:"" > label:<name:"location" value:"us-central1" > label:<name:"param1" value:"pubsub.googleapis.com" > label:<name:"param2" value:"" > label:<name:"param3" value:"v1" > label:<name:"param4" value:"google.pubsub.v1.Subscriber.Acknowledge" > label:<name:"project_id" value:"some-gcp-project" > label:<name:"quota_location" value:"us-central1" > label:<name:"quota_name" value:"pubsub.googleapis.com/regionalacknowledger" > label:<name:"unit" value:"1" > gauge:<value:1.829758e+06 > timestamp_ms:1692304080000 } was collected before with the same name and label values
* [from Gatherer #2] collected metric "stackdriver_deprecated_resource_serviceruntime_googleapis_com_reserved_metric_1" { label:<name:"credential_id" value:"" > label:<name:"location" value:"us-central1" > label:<name:"param1" value:"pubsub.googleapis.com" > label:<name:"param2" value:"" > label:<name:"param3" value:"v1" > label:<name:"param4" value:"google.pubsub.v1.Subscriber.ModifyAckDeadline" > label:<name:"project_id" value:"some-gcp-project" > label:<name:"quota_location" value:"us-central1" > label:<name:"quota_name" value:"pubsub.googleapis.com/regionalacknowledger" > label:<name:"unit" value:"1" > gauge:<value:794670 > timestamp_ms:1692304080000 } was collected before with the same name and label values
* [from Gatherer #2] collected metric "stackdriver_deprecated_resource_serviceruntime_googleapis_com_reserved_metric_1" { label:<name:"credential_id" value:"" > label:<name:"location" value:"us-central1" > label:<name:"param1" value:"pubsub.googleapis.com" > label:<name:"param2" value:"" > label:<name:"param3" value:"v1" > label:<name:"param4" value:"google.pubsub.v1.Subscriber.StreamingPull" > label:<name:"project_id" value:"some-gcp-project" > label:<name:"quota_location" value:"us-central1" > label:<name:"quota_name" value:"pubsub.googleapis.com/regionalstreamingpullconnections" > label:<name:"unit" value:"1" > gauge:<value:8499 > timestamp_ms:1692304080000 } was collected before with the same name and label values
* [from Gatherer #2] collected metric "stackdriver_deprecated_resource_serviceruntime_googleapis_com_reserved_metric_1" { label:<name:"credential_id" value:"" > label:<name:"location" value:"us-central1" > label:<name:"param1" value:"pubsub.googleapis.com" > label:<name:"param2" value:"" > label:<name:"param3" value:"v1" > label:<name:"param4" value:"google.pubsub.v1.Subscriber.StreamingPull" > label:<name:"project_id" value:"some-gcp-project" > label:<name:"quota_location" value:"us-central1" > label:<name:"quota_name" value:"pubsub.googleapis.com/regionalstreamingpullsubscriber" > label:<name:"unit" value:"1" > gauge:<value:1.3234136e+07 > timestamp_ms:1692304080000 } was collected before with the same name and label values

4wdonny commented 3 weeks ago

Wanted to post that I am having the same issue. This fully blocks the use of the exporter, as it won't serve any other metrics

4wdonny commented 3 weeks ago

I believe this would be fixed by https://github.com/prometheus-community/stackdriver_exporter/pull/319

prometheus-community / stackdriver_exporter

Getting error message has label dimensions inconsistent with previously collected metrics in the same metric family #27