search

fluxcd / flagger

Progressive delivery Kubernetes operator (Canary, A/B Testing and Blue/Green deployments)
https://docs.flagger.app
Apache License 2.0
4.88k stars 729 forks source link

no values found for istio metric request-success-rate #1365

Open mark8s opened 1 year ago

mark8s commented 1 year ago

I ran into a problem that seems to have happened to many others, the details of the problem are:

   Warning Synced 6m14s (x3 over 11m) flagger Halt advancement no values found for istio metric request-success-rate probably podinfo.test is not receiving traffic: running query failed: no values found

Why does this problem occur? How to solve it, does anyone know?

My istio version is 1.11, and prometheus is installed accordingly.

frankjkelly commented 1 year ago

For us the issue was evident when we looked in Prometheus for the appropriate metrics https://docs.flagger.app/faq#http-requests-success-rate-percentage

In our case 1) The metric istio_requests_total was not being pushed to prometheus 2) Once we fixed that we noticed the label reporter="destination" on the query was also being dropped

bakuljajan commented 2 months ago

I got the same problem but with the response_code!~"5.*" part since even with any traffic the service rarely(almost never) outputs a 5xx response code which gives us the error Halt advancement no values found for custom metric: error-rate: no values found.