open-telemetry / opentelemetry-operator

Kubernetes Operator for OpenTelemetry Collector
Apache License 2.0
1.21k stars 439 forks source link

Adding additional resource attributes on traces #2049

Open nscriptdotgo opened 1 year ago

nscriptdotgo commented 1 year ago

Hello,

I am using the operator, and have many dotnet services I would like to instrument. I have traces getting sent to Grafana Tempo, however I am trying to add the cluster name attribute via Collector CRD config and I believe the OTEL_RESOURCE_ATTRIBUTES env var that is created from the sidecar injection is totally overriding whatever I put in the config which is below (trying to insert the k8s cluster name)


  env:
    - name: KUBE_NODE_NAME
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: spec.nodeName
  config: |
    processors:
      batch:
      resource:
        attributes:
        - key: k8s.cluster.name
          from_attribute: k8s-cluster
          action: insert
service:
      extensions: [basicauth/traces,basicauth/metrics]
      pipelines:
        traces:
          receivers: [otlp]
          processors: [k8sattributes,resource]
          exporters: [logging,otlp]```

How can I make this possible? thanks
TylerHelmuth commented 1 year ago

I'm not sure how OTEL_RESOURCE_ATTRIBUTES would be affecting the collector's pipeline, though. If that variable exists on the Collector sidecar then it will affect the collector's telemetry about itself, but it will not affect the telemetry that the collector is processing.

Does the data the collector is processing actually have an attribute called k8s-cluster on it? You resourceprocessor needs that attribute to be present in order to set a new attribute called k8s.cluster.name.