open-telemetry / opentelemetry-helm-charts

OpenTelemetry Helm Charts
https://opentelemetry.io
Apache License 2.0
400 stars 492 forks source link

Can't use otlp/jaeger as exporter #1073

Open sterziev88 opened 8 months ago

sterziev88 commented 8 months ago

Hello,

I am trying to collect and send traces via opentelemetry to my jaeger. At the moment I use jaeger UI to see my traces and my applications use zipkin to send that traces. I want to use opentelemetry and the is what I have configured until now:

config: exporters: debug: verbosity: detailed sampling_initial: 5 sampling_thereafter: 200

Will be removed in a future release.

Use the debug exporter instead.

logging: {} otlp/jaeger: endpoint: "jaeger-collector.jaeger.svc.cluster.local:4317" tls: insecure: true extensions:

The health_check extension is mandatory for this chart.

Without the health_check extension the collector will fail the readiness and liveliness probes.

The health_check extension can be modified, but should never be removed.

health_check: endpoint: ${env:MY_POD_IP}:13133 memory_ballast: {} processors: batch: {}

If set to null, will be overridden with values based on k8s resource limits

memory_limiter: null receivers: jaeger: protocols: grpc: endpoint: ${env:MY_POD_IP}:14250 thrift_http: endpoint: ${env:MY_POD_IP}:14268 thrift_compact: endpoint: ${env:MY_POD_IP}:6831 otlp: protocols: grpc: endpoint: ${env:MY_POD_IP}:4317 http: endpoint: ${env:MY_POD_IP}:4318 prometheus: config: scrape_configs:

  • job_name: opentelemetry-collector scrape_interval: 10s static_configs:
  • targets:
  • ${env:MY_POD_IP}:8888 zipkin: endpoint: ${env:MY_POD_IP}:9411 service: telemetry: metrics: address: ${env:MY_POD_IP}:8888 extensions:
  • health_check
  • memory_ballast pipelines: traces: exporters:
  • debug
  • otlp/jaeger processors:
  • memory_limiter
  • batch receivers:
  • otlp
  • jaeger

but I receive an error: 2024-03-13T13:08:58.755Z warn zapgrpc/zapgrpc.go:195 [core] [Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {Addr: "jaeger-collector.jaeger.svc.cluster.local:4317", ServerName: "jaeger-collector.jaeger.svc.cluster.local:4317", }. Err: connection error: desc = "transport: Error while dialing: dial tcp 172.20.136.48:4317: connect: connection refused" {"grpc_log": true}

If add this for endpoint: tls: insecure: true I receive the same error.

Any suggestions?

kjhe11e commented 8 months ago

I'm not a maintainer and not sure what the exact issue is yet, but I was getting the same error and was able to fix using the following setup/configs. Hope it helps.

helm install otel-collector open-telemetry/opentelemetry-collector -f otel-values.yaml

otel-values.yaml ``` # Valid values are "daemonset", "deployment", and "statefulset". mode: "deployment" config: exporters: debug: verbosity: detailed otlp: endpoint: jaeger-collector.default.svc.cluster.local:4317 tls: insecure: true processors: batch: {} receivers: otlp: protocols: grpc: endpoint: ${env:MY_POD_IP}:4317 http: endpoint: ${env:MY_POD_IP}:4318 service: telemetry: metrics: address: ${env:MY_POD_IP}:8888 extensions: - health_check - memory_ballast pipelines: logs: exporters: - debug processors: - batch receivers: - otlp metrics: exporters: - debug processors: - batch receivers: - otlp traces: exporters: - debug - otlp processors: - batch receivers: - otlp ```

helm install jaeger jaegertracing/jaeger -f jaeger-values.yaml

jaeger-values.yaml ``` # Default values for jaeger. # This is a YAML-formatted file. # Jaeger values are grouped by component. Cassandra values override subchart values global: imageRegistry: provisionDataStore: cassandra: false elasticsearch: false kafka: false networkPolicy: enabled: false # Overrides the image tag where default is the chart appVersion. tag: "" nameOverride: "" fullnameOverride: "" allInOne: enabled: true replicas: 1 image: registry: "" repository: jaegertracing/all-in-one tag: "" digest: "" pullPolicy: IfNotPresent pullSecrets: [] extraEnv: [] extraSecretMounts: [] args: [] serviceAccount: annotations: {} service: headless: true collector: otlp: grpc: name: otlp-grpc http: name: otlp-http ingress: enabled: false annotations: {} labels: {} pathType: nodeSelector: {} storage: # allowed values (cassandra, elasticsearch) type: none cassandra: host: cassandra port: 9042 # Change this value to false if you want to avoid starting the # -cassandra-schema Job schemaJobEnabled: true tls: enabled: false secretName: cassandra-tls-secret user: user usePassword: true password: password keyspace: jaeger_v1_test ## Use existing secret (ignores previous password) # existingSecret: ## Cassandra related env vars to be configured on the concerned components extraEnv: [] ## Cassandra related cmd line opts to be configured on the concerned components cmdlineParams: {} elasticsearch: scheme: http host: elasticsearch-master port: 9200 anonymous: false user: elastic usePassword: true password: changeme nodesWanOnly: false extraEnv: [] ## ES related cmd line opts to be configured on the concerned components cmdlineParams: {} tls: enabled: false secretName: es-tls-secret # The mount properties of the secret mountPath: /es-tls/ca-cert.pem subPath: ca-cert.pem # How ES_TLS_CA variable will be set in the various components ca: /es-tls/ca-cert.pem kafka: brokers: - kafka:9092 topic: jaeger_v1_test authentication: none extraEnv: [] grpcPlugin: extraEnv: [] # Begin: Override values on the Cassandra subchart to customize for Jaeger cassandra: persistence: # To enable persistence, please see the documentation for the Cassandra chart enabled: false config: cluster_name: jaeger seed_size: 1 dc_name: dc1 rack_name: rack1 endpoint_snitch: GossipingPropertyFileSnitch # End: Override values on the Cassandra subchart to customize for Jaeger # Begin: Override values on the Kafka subchart to customize for Jaeger kafka: listeners: client: protocol: PLAINTEXT controller: protocol: PLAINTEXT interbroker: protocol: PLAINTEXT external: protocol: PLAINTEXT controller: replicaCount: 3 extraConfig: | auto.create.topics.enable=true # This is the jaeger-cassandra-schema Job which sets up the Cassandra schema for # use by Jaeger schema: annotations: {} image: registry: "" repository: jaegertracing/jaeger-cassandra-schema tag: "" digest: "" pullPolicy: IfNotPresent pullSecrets: [] resources: {} serviceAccount: create: true # Explicitly mounts the API credentials for the Service Account automountServiceAccountToken: true name: podAnnotations: {} podLabels: {} securityContext: {} podSecurityContext: {} ## Deadline for cassandra schema creation job activeDeadlineSeconds: 300 extraEnv: [] elasticsearch: {} ingester: enabled: false podSecurityContext: {} securityContext: {} annotations: {} image: registry: "" repository: jaegertracing/jaeger-ingester tag: "" digest: "" pullPolicy: IfNotPresent pullSecrets: [] dnsPolicy: ClusterFirst cmdlineParams: {} replicaCount: 1 autoscaling: enabled: false minReplicas: 2 maxReplicas: 10 behavior: {} service: annotations: {} # List of IP ranges that are allowed to access the load balancer (if supported) loadBalancerSourceRanges: [] type: ClusterIP resources: {} serviceAccount: create: true # Explicitly mounts the API credentials for the Service Account automountServiceAccountToken: false annotations: {} name: nodeSelector: {} tolerations: [] affinity: {} podAnnotations: {} podLabels: {} extraSecretMounts: [] extraConfigmapMounts: [] extraEnv: [] envFrom: [] initContainers: [] serviceMonitor: enabled: false additionalLabels: {} relabelings: [] metricRelabelings: [] agent: podSecurityContext: {} securityContext: {} enabled: false annotations: {} image: registry: "" repository: jaegertracing/jaeger-agent tag: "" digest: "" pullPolicy: IfNotPresent pullSecrets: [] cmdlineParams: {} extraEnv: [] daemonset: useHostPort: false updateStrategy: {} service: annotations: {} # List of IP ranges that are allowed to access the load balancer (if supported) loadBalancerSourceRanges: [] type: ClusterIP # zipkinThriftPort :accept zipkin.thrift over compact thrift protocol zipkinThriftPort: 5775 # compactPort: accept jaeger.thrift over compact thrift protocol compactPort: 6831 # binaryPort: accept jaeger.thrift over binary thrift protocol binaryPort: 6832 # samplingPort: (HTTP) serve configs, sampling strategies samplingPort: 5778 resources: {} serviceAccount: create: true # Explicitly mounts the API credentials for the Service Account automountServiceAccountToken: false name: annotations: {} nodeSelector: {} tolerations: [] affinity: {} podAnnotations: {} podLabels: {} extraSecretMounts: [] extraConfigmapMounts: [] envFrom: [] useHostNetwork: false dnsPolicy: ClusterFirst priorityClassName: "" initContainers: [] serviceMonitor: enabled: false additionalLabels: {} relabelings: [] metricRelabelings: [] collector: podSecurityContext: {} securityContext: {} enabled: false annotations: {} image: registry: "" repository: jaegertracing/jaeger-collector tag: "" digest: "" pullPolicy: IfNotPresent pullSecrets: [] dnsPolicy: ClusterFirst extraEnv: [] envFrom: [] cmdlineParams: {} basePath: / replicaCount: 1 autoscaling: enabled: false minReplicas: 2 maxReplicas: 10 behavior: {} service: annotations: {} # The IP to be used by the load balancer (if supported) loadBalancerIP: "" # List of IP ranges that are allowed to access the load balancer (if supported) loadBalancerSourceRanges: [] type: ClusterIP # Cluster IP address to assign to service. Set to None to make service headless clusterIP: "" grpc: port: 14250 # nodePort: # httpPort: can accept spans directly from clients in jaeger.thrift format http: port: 14268 # nodePort: # can accept Zipkin spans in JSON or Thrift zipkin: {} otlp: grpc: {} http: {} ingress: enabled: false annotations: {} labels: {} pathType: resources: {} serviceAccount: create: true # Explicitly mounts the API credentials for the Service Account automountServiceAccountToken: false name: annotations: {} nodeSelector: {} tolerations: [] affinity: {} podAnnotations: {} podLabels: {} extraSecretMounts: [] extraConfigmapMounts: [] priorityClassName: "" serviceMonitor: enabled: false additionalLabels: {} relabelings: [] metricRelabelings: [] initContainers: [] networkPolicy: enabled: false query: enabled: false basePath: / initContainers: [] oAuthSidecar: enabled: false resources: {} image: registry: quay.io repository: oauth2-proxy/oauth2-proxy tag: v7.6.0 digest: "" pullPolicy: IfNotPresent pullSecrets: [] containerPort: 4180 args: [] extraEnv: [] extraConfigmapMounts: [] extraSecretMounts: [] podSecurityContext: {} securityContext: {} agentSidecar: enabled: true annotations: {} image: registry: "" repository: jaegertracing/jaeger-query tag: "" digest: "" pullPolicy: IfNotPresent pullSecrets: [] dnsPolicy: ClusterFirst cmdlineParams: {} extraEnv: [] envFrom: [] replicaCount: 1 service: annotations: {} type: ClusterIP # List of IP ranges that are allowed to access the load balancer (if supported) loadBalancerSourceRanges: [] port: 80 ingress: enabled: false annotations: {} labels: {} pathType: health: exposed: false resources: {} serviceAccount: create: true # Explicitly mounts the API credentials for the Service Account automountServiceAccountToken: false name: annotations: {} nodeSelector: {} tolerations: [] affinity: {} podAnnotations: {} podLabels: {} extraConfigmapMounts: [] extraVolumes: [] sidecars: [] priorityClassName: "" serviceMonitor: enabled: false additionalLabels: {} # https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#relabelconfig relabelings: [] # -- ServiceMonitor metric relabel configs to apply to samples before ingestion # https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api.md#endpoint metricRelabelings: [] networkPolicy: enabled: false spark: enabled: false annotations: {} image: registry: "" repository: jaegertracing/spark-dependencies tag: latest digest: "" pullPolicy: IfNotPresent pullSecrets: [] cmdlineParams: {} extraEnv: [] schedule: "49 23 * * *" successfulJobsHistoryLimit: 5 failedJobsHistoryLimit: 5 concurrencyPolicy: Forbid resources: {} serviceAccount: create: true annotations: {} # Explicitly mounts the API credentials for the Service Account automountServiceAccountToken: false name: nodeSelector: {} tolerations: [] affinity: {} extraSecretMounts: [] extraConfigmapMounts: [] podAnnotations: {} podLabels: {} # ttlSecondsAfterFinished: 120 esIndexCleaner: enabled: false securityContext: runAsUser: 1000 podSecurityContext: runAsUser: 1000 annotations: {} image: registry: "" repository: jaegertracing/jaeger-es-index-cleaner tag: "" digest: "" pullPolicy: IfNotPresent pullSecrets: [] cmdlineParams: {} extraEnv: [] schedule: "55 23 * * *" successfulJobsHistoryLimit: 3 failedJobsHistoryLimit: 3 concurrencyPolicy: Forbid resources: {} numberOfDays: 7 serviceAccount: create: true annotations: {} # Explicitly mounts the API credentials for the Service Account automountServiceAccountToken: false name: nodeSelector: {} tolerations: [] affinity: {} extraSecretMounts: [] extraConfigmapMounts: [] podAnnotations: {} podLabels: {} # ttlSecondsAfterFinished: 120 esRollover: enabled: false securityContext: {} podSecurityContext: runAsUser: 1000 annotations: {} image: registry: "" repository: jaegertracing/jaeger-es-rollover tag: "" digest: "" pullPolicy: IfNotPresent pullSecrets: [] cmdlineParams: {} extraEnv: - name: CONDITIONS value: '{"max_age": "1d"}' schedule: "10 0 * * *" successfulJobsHistoryLimit: 3 failedJobsHistoryLimit: 3 concurrencyPolicy: Forbid resources: {} serviceAccount: create: true # Explicitly mounts the API credentials for the Service Account automountServiceAccountToken: false name: nodeSelector: {} tolerations: [] affinity: {} extraSecretMounts: [] extraConfigmapMounts: [] podAnnotations: {} podLabels: {} # ttlSecondsAfterFinished: 120 initHook: extraEnv: [] annotations: {} podAnnotations: {} podLabels: {} ttlSecondsAfterFinished: 120 esLookback: enabled: false securityContext: {} podSecurityContext: runAsUser: 1000 annotations: {} image: registry: "" repository: jaegertracing/jaeger-es-rollover tag: "" digest: "" pullPolicy: IfNotPresent pullSecrets: [] cmdlineParams: {} extraEnv: - name: UNIT value: days - name: UNIT_COUNT value: "7" schedule: "5 0 * * *" successfulJobsHistoryLimit: 3 failedJobsHistoryLimit: 3 concurrencyPolicy: Forbid resources: {} serviceAccount: create: true annotations: {} # Explicitly mounts the API credentials for the Service Account automountServiceAccountToken: false name: nodeSelector: {} tolerations: [] affinity: {} extraSecretMounts: [] extraConfigmapMounts: [] podAnnotations: {} podLabels: {} # Array with extra yaml objects to install alongside the chart. Values are evaluated as a template. extraObjects: [] ```

My backend service is using the OTLP exporter via gRCP, similar to https://github.com/open-telemetry/opentelemetry-go/blob/main/example/otel-collector/main.go

JaredTan95 commented 7 months ago

@sterziev88 Obviously, the jaeger is not connected, you need to check your jaeger address is correct