This is my first attempt to write a new way of listing out instances because scrape times are >1m when using the AggregatedList method. Introduced a ListInstancesInZone method that accepts a projectID, zone,and compute.Service. This will effectively do the same thing as ListInstances, however for a particular zone. The largest zones still take upwards of 30 seconds, but since we run them in parallel we're bound to the slowest call instead of aggregating them all together.
Updated gke.go to list out the zones and then use a channel to send instances to.
We run all zones in parallel and read the results as they are published to the channel.
Then from there we iterate over instances and send the metrics to the channel passed in from Prometheus.
Followed the same strategy for Compute. There's potential here reduce duplicated code, but I think where it's at right now is just fine as it allows us to expand either implementation cleanly.
This is my first attempt to write a new way of listing out instances because scrape times are >1m when using the AggregatedList method. Introduced a ListInstancesInZone method that accepts a projectID, zone,and compute.Service. This will effectively do the same thing as ListInstances, however for a particular zone. The largest zones still take upwards of 30 seconds, but since we run them in parallel we're bound to the slowest call instead of aggregating them all together.
Updated
gke.goto list out the zones and then use a channel to send instances to. We run all zones in parallel and read the results as they are published to the channel. Then from there we iterate over instances and send the metrics to the channel passed in from Prometheus.Followed the same strategy for Compute. There's potential here reduce duplicated code, but I think where it's at right now is just fine as it allows us to expand either implementation cleanly.