jaegertracing / helm-charts

Helm Charts for Jaeger backend
Apache License 2.0
265 stars 340 forks source link

Jaeger Production Strategy - Collector Not Processing Spans #262

Open dw-develop opened 3 years ago

dw-develop commented 3 years ago

I am not able to upload span tags to the zipkin port 9411 endpoint. I am using elastic search backend. It seems the collector takes the sample span tag but then nothing processes it.

To Reproduce Steps to reproduce the behavior:

  1. Deploy Jaeger HELM Chart.
    
    kubectl create namespace observability

kubectl create configmap jaeger-tls --from-file=./es.crt -n observability

helm repo add jaegertracing https://jaegertracing.github.io/helm-charts

helm upgrade --install jaeger jaegertracing/jaeger -f ./jaeger-helm.yaml -n observability --wait --insecure-skip-tls-verify=true


2. Port forward to collector: 
`kubectl port-forward svc/jaeger-collector -n observability 9411:9411`
3. POST sample span. Returns response 202.
4. Navigate to Jaeger UI. The new span does not show up. Nothing in logs indicating a failure.

**Expected behavior**
A clear and concise description of what you expected to happen.

**Version (please complete the following information):**
 - OS: Ubuntu
 - Jaeger version: Latest
 - Deployment: AKS: Kubernetes

**What troubleshooting steps did you try?**
I have tried troubleshooting connectivity but nothing shows up in logs.

http://localhost:9411/api/v2/spans
Response: 202 Accepted
Sample POST

[ { "id": "352bff9a74ca9ad2", "traceId": "5af7183fb1d4cf5f", "parentId": "6b221d5bc9e6496c", "name": "get /api", "timestamp": 1556604172355737, "duration": 1431, "kind": "SERVER", "localEndpoint": { "serviceName": "backend", "ipv4": "192.168.99.1", "port": 3306 }, "remoteEndpoint": { "ipv4": "172.19.0.2", "port": 58648 }, "tags": { "http.method": "GET", "http.path": "/api" } } ]


HELM Chart Values Override

storage: type: elasticsearch elasticsearch: host: myhost.com port: 443 scheme: https user: elastic usePassword: true password: provisionDataStore: cassandra: false elasticsearch: false query: basePath: /jaeger cmdlineParams: es.tls.ca: "/tls/es.crt" extraConfigmapMounts:

dw-develop commented 3 years ago

Can this be assigned for review?

dw-develop commented 3 years ago

I was able to get the default cassandra deployment to work. The elasticsearch default settings do not allow the jaeger UI to access elasticsearch or something is wrong when it creates new indexes on initialization.