search

jaegertracing / jaeger-operator

Jaeger Operator for Kubernetes simplifies deploying and running Jaeger on Kubernetes.
https://www.jaegertracing.io/docs/latest/operator/
Apache License 2.0
1.03k stars 345 forks source link

[Bug]: Operator showing error with injection sidecar #2105

Open minh-oolio opened 2 years ago

minh-oolio commented 2 years ago

What happened?

hello team, I implemented sidecar injected on deployment as this document https://github.com/jaegertracing/jaeger-operator/blob/main/examples/business-application-injected-sidecar.yaml and found error message from jaeger operator 

1.6672863326932034e+09  INFO    annotation present on deployment    {"namespace": "applicationA", "deployment": "accounts"}
1.6672863326933312e+09  INFO    no suitable Jaeger instances found to inject a sidecar  {"namespace": "applicationA"}

Steps to reproduce

  1. deployed jaeger and jaeger operator by helm chart with latest version (namespace jaeger)
  2. implement sidecar inject on deployment of application (other namespace applicationA )
  3. we are using traefik as service mesh https://traefik.io/blog/application-request-tracing-with-traefik-and-jaeger-on-kubernetes/
  4. check log operator and found error message no suitable Jaeger instances found to inject a sidecar

here is services of jaeger was deployed image

Expected behavior

Able to tracing appliacation api. something like this picture image

Relevant log output

Operator pod logs

1.6672916883622577e+09  INFO    no suitable Jaeger instances found to inject a sidecar  {"namespace": "applicationA"}

1.6672916883625038e+09  DEBUG   controller-runtime.webhook.webhooks wrote response  {"webhook": "/mutate-v1-deployment", "code": 200, "reason": "no suitable Jaeger instances found to inject a sidecar", "UID": "294a91d5-fe16-417e-a19c-c3ac5664599e", "allowed": true}

Screenshot

No response

Additional context

No response

Jaeger backend version

v1.37.0

SDK

No response

Pipeline

No response

Stogage backend

No response

Operating system

No response

Deployment model

kubernetes (EKS)

Deployment configs

No response

frzifus commented 2 years ago

Seems your deployment needs an injection. But can not find a Jaeger instance in the same namespace. In that case your sidecar doesnt know where it should send the collected data.

Have a look here:

minh-oolio commented 2 years ago

hi @frzifus thanks for your response Is there any solution for my case? I deployed jaeger and jaeger operatorin jaeger namespace and tracing application on other namespace. thank you very much

frzifus commented 2 years ago

Hi @minh-oolio, from my understanding you need to define WATCH_NAMESPACES in your operator.yaml file. To make sure the right jaeger instance is picked, change your annotation value to your instance name.

kind: Deployment
metadata:
  name: myapp
  annotations:
    "sidecar.jaegertracing.io/inject": "<INSTANCE_NAME>"

On the operator side, your jaeger instance should then be found here.

minh-oolio commented 2 years ago

hi @frzifus thanks a lot for your suggestion it seem it able to inject sidecar to pod but it still trace on the console i am concern, do i need to change jaeger-operator-jaeger to jaeger-operator-jaeger.jaeger.svc please advise on it

k get jaegers
NAME                     STATUS    VERSION   STRATEGY   STORAGE   AGE
jaeger-operator-jaeger   Running   1.38.0    allinone   memory    3h39m

with this annotaion 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gateway
  annotations:
    sidecar.jaegertracing.io/inject: jaeger-operator-jaeger

and here is a log of jageragent injected to gateway pod 

31
2022/11/08 00:46:53 maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined
30
{"level":"info","ts":1667868413.1804988,"caller":"flags/service.go:119","msg":"Mounting metrics handler on admin server","route":"/metrics"}
29
{"level":"info","ts":1667868413.180543,"caller":"flags/service.go:125","msg":"Mounting expvar handler on admin server","route":"/debug/vars"}
28
{"level":"info","ts":1667868413.1809232,"caller":"flags/admin.go:129","msg":"Mounting health check on admin server","route":"/"}
27
{"level":"info","ts":1667868413.1809716,"caller":"flags/admin.go:143","msg":"Starting admin HTTP server","http-addr":":14271"}
26
{"level":"info","ts":1667868413.1809838,"caller":"flags/admin.go:121","msg":"Admin server started","http.host-port":"[::]:14271","health-status":"unavailable"}
25
{"level":"info","ts":1667868413.1815352,"caller":"grpc/builder.go:73","msg":"Agent requested insecure grpc connection to collector(s)"}
24
{"level":"info","ts":1667868413.1815639,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel created","system":"grpc","grpc_log":true}
23
{"level":"info","ts":1667868413.1815975,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] original dial target is: \"dns:///jaeger-operator-jaeger-collector-headless.jaeger.svc:14250\"","system":"grpc","grpc_log":true}
22
{"level":"info","ts":1667868413.181634,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] parsed dial target is: {Scheme:dns Authority: Endpoint:jaeger-operator-jaeger-collector-headless.jaeger.svc:14250 URL:{Scheme:dns Opaque: User: Host: Path:/jaeger-operator-jaeger-collector-headless.jaeger.svc:14250 RawPath: OmitHost:false ForceQuery:false RawQuery: Fragment: RawFragment:}}","system":"grpc","grpc_log":true}
21
{"level":"info","ts":1667868413.1816468,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel authority set to \"jaeger-operator-jaeger-collector-headless.jaeger.svc:14250\"","system":"grpc","grpc_log":true}
20
{"level":"info","ts":1667868413.1829572,"caller":"grpc/builder.go:113","msg":"Checking connection to collector"}
19
{"level":"info","ts":1667868413.182981,"caller":"grpc/builder.go:124","msg":"Agent collector connection state change","dialTarget":"dns:///jaeger-operator-jaeger-collector-headless.jaeger.svc:14250","status":"IDLE"}
18
{"level":"info","ts":1667868413.183288,"caller":"./main.go:88","msg":"Starting agent"}
17
{"level":"info","ts":1667868413.1833262,"caller":"healthcheck/handler.go:129","msg":"Health Check state change","status":"ready"}
16
{"level":"info","ts":1667868413.1833835,"caller":"app/agent.go:69","msg":"Starting jaeger-agent HTTP server","http-port":5778}
15
{"level":"info","ts":1667868413.1956995,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Resolver state updated: {\n  \"Addresses\": [\n    {\n      \"Addr\": \"192.168.112.106:14250\",\n      \"ServerName\": \"\",\n      \"Attributes\": null,\n      \"BalancerAttributes\": null,\n      \"Type\": 0,\n      \"Metadata\": null\n    }\n  ],\n  \"ServiceConfig\": null,\n  \"Attributes\": null\n} (resolver returned new addresses)","system":"grpc","grpc_log":true}
14
{"level":"info","ts":1667868413.1957407,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel switches to new LB policy \"round_robin\"","system":"grpc","grpc_log":true}
13
{"level":"info","ts":1667868413.1957922,"caller":"grpclog/component.go:55","msg":"[balancer]base.baseBalancer: got new ClientConn state: {{[{\n  \"Addr\": \"192.168.112.106:14250\",\n  \"ServerName\": \"\",\n  \"Attributes\": null,\n  \"BalancerAttributes\": null,\n  \"Type\": 0,\n  \"Metadata\": null\n}] <nil> <nil>} <nil>}","system":"grpc","grpc_log":true}
12
{"level":"info","ts":1667868413.1958196,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel created","system":"grpc","grpc_log":true}
11
{"level":"info","ts":1667868413.1958375,"caller":"grpclog/component.go:71","msg":"[roundrobin]roundrobinPicker: Build called with info: {map[]}","system":"grpc","grpc_log":true}
10
{"level":"info","ts":1667868413.1958477,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
9
{"level":"info","ts":1667868413.1958697,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
8
{"level":"info","ts":1667868413.1958842,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel picks a new address \"192.168.112.106:14250\" to connect","system":"grpc","grpc_log":true}
7
{"level":"info","ts":1667868413.195938,"caller":"grpc/builder.go:124","msg":"Agent collector connection state change","dialTarget":"dns:///jaeger-operator-jaeger-collector-headless.jaeger.svc:14250","status":"CONNECTING"}
6
{"level":"info","ts":1667868413.1959598,"caller":"grpclog/component.go:71","msg":"[balancer]base.baseBalancer: handle SubConn state change: 0xc00036ab50, CONNECTING","system":"grpc","grpc_log":true}
5
{"level":"info","ts":1667868413.2003353,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1 SubChannel #2] Subchannel Connectivity change to READY","system":"grpc","grpc_log":true}
4
{"level":"info","ts":1667868413.200361,"caller":"grpclog/component.go:71","msg":"[balancer]base.baseBalancer: handle SubConn state change: 0xc00036ab50, READY","system":"grpc","grpc_log":true}
3
{"level":"info","ts":1667868413.2003944,"caller":"grpclog/component.go:71","msg":"[roundrobin]roundrobinPicker: Build called with info: {map[0xc00036ab50:{{\n  \"Addr\": \"192.168.112.106:14250\",\n  \"ServerName\": \"\",\n  \"Attributes\": null,\n  \"BalancerAttributes\": null,\n  \"Type\": 0,\n  \"Metadata\": null\n}}]}","system":"grpc","grpc_log":true}
2
{"level":"info","ts":1667868413.2004066,"caller":"channelz/funcs.go:340","msg":"[core][Channel #1] Channel Connectivity change to READY","system":"grpc","grpc_log":true}
1
{"level":"info","ts":1667868413.2004158,"caller":"grpc/builder.go:124","msg":"Agent collector connection state change","dialTarget":"dns:///jaeger-operator-jaeger-collector-headless.jaeger.svc:14250","status":"READY"}

image