search

grafana / k8s-monitoring-helm

Apache License 2.0
225 stars 86 forks source link

Mismatch between Beyla and Loki pod logs on `service.name`, `service.namespace`, and `service.instance.id` #942

Open cyrille-leclerc opened 1 day ago

cyrille-leclerc commented 1 day ago
  TRACE BY BEYLA LOGS BY GRAFANA K8S MONITORING HELM CHART
host.id 8b3ab3487d2d4b3d9ffa0b93accc6b78  
host.name authentication-deployment-7d77f7886b-6kwhn  
job   dev/authentication-container
k8s.cluster.name dev  dev
k8s.container.name  authentication-container
k8s.deployment.name authentication-deployment  
k8s.namespace.name dev  dev
k8s.node.name atlubun2devn2  
k8s.pod.name authentication-deployment-7d77f7886b-6kwhn  
k8s.pod.start_time 2024-11-07 19:53:02 +0000 UTC  
k8s.pod.uid 4caaca67-4386-4148-919c-1615e823cece  
k8s.replicaset.name authentication-deployment-7d77f7886b  
otel.library.name github.com/grafana/beyla  
pod   authentication-deployment-7d77f7886b-6kwhn
🔴 service.instance.id beyla-ld92z-2310310  
🔴 service.name authentication-deployment  authentication-container
🔴 service.namespace dev  
telemetry.sdk.language go  
telemetry.sdk.name beyla  
petewall commented 1 day ago

Trying to determine how to resolve this. service.namespace is obvious. service.name appears to be the name of the deployment, where loki source only sees pods and containers. service.instance.id appears to be the Beyla pod. Is that even the right setting? What should it be?

cyrille-leclerc commented 1 day ago

Thanks @petewall . I started brainstorming with @zeitlinger and plan to involve next week the Beyla team.