envoyproxy / envoy

Cloud-native high-performance edge/middle/service proxy
https://www.envoyproxy.io
Apache License 2.0
24.9k stars 4.79k forks source link

Envoy 1.31.2 not generating cluster.outbound metrics #36565

Open rwincewicz opened 6 days ago

rwincewicz commented 6 days ago

Title: Envoy 1.31.2 not generating cluster.outbound metrics

Description: I am seeing that the cluster.outbound metrics that used to be generated by Envoy are now missing having upgraded from 1.30.6 to 1.31.2. The Envoy upgrade has come from upgrading Istio from 1.22.5 to 1.23.3.

Repro steps:

Deploy a container in Kubernetes with an Istio sidecar. Istio is deployed via helm charts.

Query the Envoy metrics for the pod using kubectl exec -n <NAMESPACE> <POD_NAME> -c istio-proxy -- pilot-agent request GET stats Observe that in Envoy version 1.30.6 there were metrics for cluster.outbound

cluster.outbound|33627||remote-service-33627.com.bind_errors: 0
cluster.outbound|33627||remote-service-33627.com.ssl.connection_error: 0
cluster.outbound|33627||remote-service-33627.com.ssl.fail_verify_error: 0
cluster.outbound|33627||remote-service-33627.com.ssl.fail_verify_no_cert: 0
cluster.outbound|33627||remote-service-33627.com.ssl.fail_verify_san: 0
cluster.outbound|33627||remote-service-33627.com.upstream_cx_protocol_error: 0

In Envoy version 1.31.2 these metrics do not appear despite the whole deployment being exactly the same. The only difference being the version of Istio and Envoy.

Admin and Stats Output: /stats - envoy-stats.txt /clusters - envoy-clusters.txt /server_info - envoy-server_info.txt

Config: envoy-rev.json

Logs: I don't have access to the separate Envoy logs but these are the logs from the istio-proxy sidecar container that contains the Envoy logs. envoy-log.txt

Call Stack: N/A

tyxia commented 2 days ago

@rwincewicz we have just released 1.32. I am wondering if you still see this issue with latest release