No metrics from compute_engine

[baboune]

Hi,

Using latest stackdriver_exporter, I seem to have ran into an issue extracting metrics from Stackdriver.

I use a service account with the Monitoring Viewer role that I inject into a docker container.

docker run --rm -it -p 9255:9255  \
  -e GOOGLE_APPLICATION_CREDENTIALS=/certs/flames-dev-monitoring-stackdriver-sa.json \
  -e STACKDRIVER_EXPORTER_GOOGLE_PROJECT_ID="flames-dev" \
  -e STACKDRIVER_EXPORTER_MONITORING_METRICS_TYPE_PREFIXES="compute.googleapis.com/instance,pubsub.googleapis.com/subscription,redis.googleapis.com/stats" \
  -e STACKDRIVER_EXPORTER_MONITORING_METRICS_INTERVAL="1m" \
  -v /local/git/flames/flames-infrastructure-k8s-addons/workspace/json-sa/:/certs/ \
  frodenas/stackdriver-exporter --log.level="debug"

On port 9255/metrics, I can see that many pubsub metrics are available. E.g.

stackdriver_pubsub_subscription_pubsub_googleapis_com_subscription_num_retained_acked_messages{project_id="flames-dev",subscription_id="flames-service-event%us-central1%event-source-subscription",unit="1"} 0
stackdriver_pubsub_subscription_pubsub_googleapis_com_subscription_num_retained_acked_messages{project_id="flames-dev",subscription_id="flames-service-subscription%no-region%event-source-subscription",unit="1"} 0
stackdriver_pubsub_subscription_pubsub_googleapis_com_subscription_num_retained_acked_messages{project_id="flames-dev",subscription_id="gcf-cloudBuildSlackIntegration-cloud-builds",unit="1"} 0
stackdriver_pubsub_subscription_pubsub_googleapis_com_subscription_num_retained_acked_messages{project_id="flames-dev",subscription_id="gcf-orchestrationMWInitialStepTransform-orchestration.mw_step_convert",unit="1"} 0
stackdriver_pubsub_subscription_pubsub_googleapis_com_subscription_num_retained_acked_messages{project_id="flames-dev",subscription_id="gcr-triggers",unit="1"} 0
stackdriver_pubsub_subscription_pubsub_googleapis_com_subscription_num_retained_acked_messages{project_id="flames-dev",subscription_id="pp_mp_file_sub",unit="1"} 0
stackdriver_pubsub_subscription_pubsub_googleapis_com_subscription_num_retained_acked_messages{project_id="flames-dev",subscription_id="spinnaker",unit="1"} 0
stackdriver_pubsub_subscription_pubsub_googleapis_com_subscription_num_retained_acked_messages{project_id="flames-dev",subscription_id="spinnaker-gcr",unit="1"} 0

There are no errors from the stackdriver_exporter prometheus counter:

# HELP stackdriver_monitoring_last_scrape_error Whether the last metrics scrape from Google Stackdriver Monitoring resulted in an error (1 for error, 0 for success).
# TYPE stackdriver_monitoring_last_scrape_error gauge
stackdriver_monitoring_last_scrape_error{project_id="flames-dev"} 0

But, there are no metrics from either redis or compute.

I also tried with the example in the stackdriver_exporter, --monitoring.metrics-type-prefixes "compute.googleapis.com/instance/cpu,compute.googleapis.com/instance/disk" and do not get any values or instances.

What am I doing wrong?

[pie-r]

same problem on:

- name: STACKDRIVER_EXPORTER_MONITORING_METRICS_TYPE_PREFIXES
              value: "cloudsql.googleapis.com"

But Setting STACKDRIVER_EXPORTER_MONITORING_METRICS_INTERVAL="2m" it works.

I don't know why 1m gives no metrics. (no errors in log)

this issue is related: https://github.com/prometheus-community/stackdriver_exporter/issues/32

[SuperQ]

The default metrics interval is 5 minutes. I'm guessing 1m is too fast.

[pie-r]

@SuperQ where I can see that the default is 5m? is due to cloudsql? I already know that the exporter has 5m as default.

[SuperQ]

5m is the exporter default. I don't know why it's the default. The change to 5m was made with no additional comment0.

[TamasNeumer]

same problem on:

- name: STACKDRIVER_EXPORTER_MONITORING_METRICS_TYPE_PREFIXES
              value: "cloudsql.googleapis.com"

But Setting STACKDRIVER_EXPORTER_MONITORING_METRICS_INTERVAL="2m" it works.

I don't know why 1m gives no metrics. (no errors in log)

this issue is related:

32

Hi,

does it still work for you? I get metrics for "gce", but for cloudsql not... I'm using the latest `version="0.11.0"...

Update:

• I managed to enable debug log level by passing the "--log.level", "debug" flag in my k8s deployment
• Metrics popped up for me. I didn't have the metrics cause of a misconfiguration made on our side. This however might help somebody else: The problem was that we have a "monitoring" GCP project through which we access the postgres databases located in different GCP projects, however the GCP project where stackdriver-exporter was running didn't have any CloudSQL databases => hence no DB metrics popped up. We grantedmonitoring.viewer in the monitoring project to the Googel Service Account (in an other project) which was used by the stackdriver-exporter. The metrics immediately showed up.