search

grafana / beyla

eBPF-based autoinstrumentation of web applications and network metrics
https://grafana.com/oss/beyla-ebpf/
Apache License 2.0
1.19k stars 77 forks source link

rpc server address, should use otel.SpanHost instead of otel.SpanPeer in function labelValuesGRPC #778

Closed corneliu-calciu closed 3 weeks ago

corneliu-calciu commented 3 weeks ago

The bug: rpc_system="grpc", server_address="backend-go", service_name="backend-go"

Actually server address is 10.152.183.18 port 50051. See below: [10.1.221.189 as backend-go]->[10.152.183.18 as 10.152.183.18:50051]

The server_address is wrong! Please see pkg/internal/export/prom/prom.go function: labelValuesHTTPClient is using otel.SpanHost(span) for the server address.

The logs: 2024-04-23 16:28:17.42342817 (148.711µs[101.119µs]) GRPC_SRV 0 /calculator.Calculator/SquareRoot [10.1.221.189 as backend-go]->[10.1.221.185 as ms-grpc-2:50051] size:0B svc=[default/ms-grpc-2 go] traceparent=[00-3d8c63d1f1cbbc4c3105477eaab79441-ee8936341d8f8576-01] 2024-04-23 16:28:17.42342817 (2.571143ms[2.571143ms]) GRPC_CLNT 0 /calculator.Calculator/SquareRoot [10.1.221.189 as backend-go]->[10.152.183.18 as 10.152.183.18:50051] size:0B svc=[default/backend-go go] traceparent=[00-3d8c63d1f1cbbc4c3105477eaab79441-374ff4797c48003f-01]

The exported metric: rpc_client_duration_seconds_count{instance="198.18.0.101:8999", job="beyladev", k8s_deployment_name="backend-go", k8s_namespace_name="apps", k8s_node_name="devenv", k8s_pod_name="backend-go-5d885f695c-cm7pq", k8s_pod_start_time="2024-04-23 16:20:23 +0000 UTC", k8s_pod_uid="53c19772-6339-4da5-93e3-85001f44f88f", k8s_replicaset_name="backend-go-5d885f695c", rpc_grpc_status_code="0", rpc_method="/calculator.Calculator/SquareRoot", rpc_system="grpc", server_address="backend-go", service_name="backend-go", service_namespace="default", target_instance="devenv-1331905"}

After the fix: rpc_client_duration_seconds_count{instance="198.18.0.101:8999", job="beyladev", k8s_deployment_name="backend-go", k8s_namespace_name="apps", k8s_node_name="devenv", k8s_pod_name="backend-go-5d885f695c-cm7pq", k8s_pod_start_time="2024-04-23 16:20:23 +0000 UTC", k8s_pod_uid="53c19772-6339-4da5-93e3-85001f44f88f", k8s_replicaset_name="backend-go-5d885f695c", rpc_grpc_status_code="0", rpc_method="/calculator.Calculator/SquareRoot", rpc_system="grpc", server_address="10.152.183.18", service_name="backend-go", service_namespace="default", target_instance="devenv-1331905"} rpc_system="grpc", server_address="10.152.183.18", service_name="backend-go"

Now the destination/server address is correct.

CLAassistant commented 3 weeks ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

:white_check_mark: corneliu-calciu
:x: corneliucalciu
You have signed the CLA already but the status is still pending? Let us recheck it.

grcevski commented 3 weeks ago

Hi @corneliu-calciu, you are absolutely right! Thanks for catching this. I think we need to split the two GRPC values into separate functions, just like we do for HTTP. In case of a client the address needs to be Host and in case of a server it needs to be Peer.