search

buildkite / buildkite-agent-metrics

A command-line tool (and Lambda) for collecting Buildkite agent metrics
MIT License
67 stars 51 forks source link

Stackdriver: Unable to read metrics from HPA #273

Open dmoxyeze opened 3 months ago

dmoxyeze commented 3 months ago

I recently deployed the buildkite-agent-metrics on GKE. I can see the metrics when I check the logs. However, when I try to scale our pods using the metrics, I get this error 

FailedGetPodsMetric:  the HPA was unable to compute the replica count: unable to get metric custom.googleapis.com|buildkite|xsplit_web|WaitingJobsCount: │
│  no metrics returned from custom metrics API

When I run kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/<namespace>/custom.googleapis.com|buildkite|project|WaitingJobsCount" | jq, I get this error:

Error from server (ServiceUnavailable): the server is currently unable to handle the request

DrJosh9000 commented 3 months ago

Hi @dmoxyeze, I can see this has also been raised through Support, we'll follow up there.