linkerd zipkin pod (container) error

[prasenforu]

Env : kubernetes 1.7.0

Using linkerd-zipkin yml file (https://github.com/linkerd/linkerd-examples/blob/master/k8s-daemonset/k8s/linkerd-zipkin.yml)

oc get all output in default namespace:

root@ip-172-31-29-6:~/zipkin# oc get all
NAME                        READY     STATUS    RESTARTS   AGE
po/l5d-f8193                2/2       Running   0          4m
po/mongo-2011101372-c92qq   1/1       Running   0          1h
po/myemp-2587386090-mxx9q   1/1       Running   0          1h
po/zipkin-274mz             1/1       Running   0          1h

NAME        DESIRED   CURRENT   READY     AGE
rc/zipkin   1         1         1         1h

NAME                   CLUSTER-IP       EXTERNAL-IP   PORT(S)                                        AGE
svc/kubernetes         10.96.0.1        <none>        443/TCP                                        2h
svc/l5d                10.104.38.74     <pending>     4140:30428/TCP,4141:30973/TCP,9990:32462/TCP   4m
svc/mongo              10.99.95.4       <none>        27017/TCP                                      1h
svc/myemp              10.99.64.252     <none>        80/TCP                                         1h
svc/zipkin             10.111.128.125   <pending>     80:32418/TCP                                   1h
svc/zipkin-collector   10.100.55.82     <none>        9410/TCP                                       1h

NAME           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/mongo   1         1         1            1           1h
deploy/myemp   1         1         1            1           1h

NAME                  DESIRED   CURRENT   READY     AGE
rs/mongo-2011101372   1         1         1         1h
rs/myemp-2587386090   1         1         1         1h

output of pod (l5d -container) logs (root@ip-172-31-29-6:~oc logs -f po/l5d-f8193 l5d)

I 0716 10:48:18.356 UTC THREAD10: k8s initializing default
E 0716 10:48:18.362 UTC THREAD17: k8s failed to list endpoints
io.buoyant.k8s.Api$UnexpectedResponse: Response("HTTP/1.1 Status(403)"): User "system:serviceaccount:default:default" cannot list endpoints in the namespace "default".
        at io.buoyant.k8s.Api$.parse(Api.scala:72)

looks like services are not discovered by linkerd pod

by searching url (https://github.com/linkerd/linkerd/issues/1347) error gone

but NO tracing in zipkin :( but in linkerd admin console I am getting incoming/outgoing data.

Not sure if zipkins required cluster role binding ?

Please help

[klingerf]

Hey @prasenforu, thanks for reporting. This sounds like a duplicate of linkerd/linkerd#1347. If linkerd is unable to route the request, then it also won't show up in the zipkin ui. If linkerd can't connect to zipkin, then you'll see errors in the logs that look like this:

I 0707 20:02:06.374 UTC THREAD111 TraceId:77689a2c404454a5: FailureAccrualFactory marking connection to "zipkin-tracer" as dead. Remote Address: Inet(localhost/127.0.0.1:1463,Map())

Absent those errors, then I think the issue is the fact that linkerd needs additional RBAC privileges to route within your cluster. There are a few suggestions of how to fix in linkerd/linkerd#1347.

[prasenforu]

Thanks 😀