search

prometheus-community / stackdriver_exporter

Google Stackdriver Prometheus exporter
Apache License 2.0
254 stars 97 forks source link

appengine.googleapis.com causes errors in prometheus handler #38

Open jmif opened 5 years ago

jmif commented 5 years ago

Hello,

When we include appengine.googleapis.com in our prefix list we see 500s being returned from the /metrics endpoint. We've turned debug logging on and it seems to be loading the app engine metrics without error:

time="2018-10-18T19:02:01Z" level=debug msg="Listing Google Stackdriver Monitoring metric descriptors starting with `appengine.googleapis.com`..." source="monitoring_collector.go:210"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/system/network/sent_bytes_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/http/server/quota_denial_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/system/memory/usage`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/system/network/received_bytes_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/http/server/response_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/http/server/dos_intercept_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/flex/disk/write_bytes_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/http/server/response_latencies`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/memcache/centi_mcu_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/memcache/operation_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/memcache/received_bytes_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/memcache/sent_bytes_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/system/cpu/usage`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/system/instance_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/flex/cpu/reserved_cores`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/flex/cpu/utilization`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/flex/disk/read_bytes_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/flex/network/sent_bytes_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/flex/network/received_bytes_count`..." source="monitoring_collector.go:166"
time="2018-10-18T19:02:01Z" level=debug msg="Retrieving Google Stackdriver Monitoring metrics for descriptor `appengine.googleapis.com/http/server/response_style_count`..." source="monitoring_collector.go:166"

There are no error logs coming from the prometheus handler (or any error logs coming in general) and I can't seem to find a way to turn on more detailed logging for the handler (new to Go so may be missing something).

Any idea what could be causing this?

Thanks for a great project, we use this in many of our Kubernetes deployments and its been very helpful.

arnisoph commented 3 years ago

Just FYI: We haven't seen 500s yet but we're missing appengine metrics too.

As this project doesn't seem to be much actively maintained, we'll have a deeper look ourselves..