search

googleapis / google-cloud-python

Google Cloud Client Library for Python
https://googleapis.github.io/google-cloud-python/
Apache License 2.0
4.81k stars 1.52k forks source link

Unable to get all monitoring metrics for num_undelivered_messages #3176

Closed samskillman closed 7 years ago

samskillman commented 7 years ago

I've written a simple utility to monitor a bunch of pubsub queues, in particular the number of undelivered messages. I'm currently unable to page through all the subscriptions and am instead only able to see 32 results. 

    client = monitoring.Client()
    q = client.query(
        "pubsub.googleapis.com/subscription/num_undelivered_messages",
        minutes=30
    )
    print len(list(q))  # Always results in 32, I have more than that.

I've also tried using the select_resource to filter the query, but that seems to only filter on the first 32 results. I'm using python2.7 and google-cloud-monitoring==0.23.0

dhermes commented 7 years ago

@samskillman I am trying to reproduce, any suggestions for reproducing? If you make the same query in the APIs explorer can you get more than 32?

samskillman commented 7 years ago

@dhermes thanks I went through https://developers.google.com/apis-explorer/#search/monitoring/cloudmonitoring/v2beta2/cloudmonitoring.timeseries.list and got the same result. I think my confusion came from a lag between when the subscription started processing messages and when the monitoring metrics showed up (which they now do as expected). I was used to the 3-5 minute lag between a new subcription and it showing up in both the stackdriver monitoring interface and through the API call in google-cloud-python, but this seemed to be more on > 30 minute lag. Probably can close the issue for now since it doesn't apply only to google-cloud-python, but the time delay between the creation of the subscription and it showing up through the monitoring API definitely caused quite a bit of confusion. Maybe worth a note somewhere in the docs if it is a reproducible/expected delay? I have used this api quite a bit and this was the first time I ran into the delay.

Thanks again!

[edit] Part of the confusion is that subscriptions that haven't processed any messages recently just don't show up at all rather than having "0" unacknowledged messages. Also realized I should have said "unacknowledged" rather than "undelivered" in the original issue.

dhermes commented 7 years ago

Thanks for confirming!

@lukesneeringer @jonparrott Where should we redirect this feedback?