kgunjikar commented 5 years ago

I'm trying to inject a sidecar, using which I want to create some CRDs. I need the sidecar to come up "in-cluster". However, I don't see a service account token getting mapped in the injected sidecar.

I'm using the configmap provided in the docs folder. With these lines added:

data: test1: | name: test1 env:

name: HELLO value: world
name: TEST value: test_that volumeMounts:
- name: test-vol mountPath: /tmp/test volumes:
- name: test-vol configMap: name: test-config serviceAccount: k8s-sidecar-injector serviceAccountName: k8s-sidecar-injector automountServiceAccountToken: true

I have cherry picked the serviceAccount related PRs.

A brief description of your problem, here, please!

2019-10-18T14:34:30.401690495-07:00 stderr F I1018 21:34:30.401614       1 webhook.go:494] AdmissionResponse: patch=[{"op":"add","path":"/spec/containers/-","value":{"name":"sidecar-wiper","image":"d
iamanti/wiper:0.2","ports":[{"containerPort":80}],"env":[{"name":"ENV_IN_SIDECAR","value":"test-in-sidecar"},{"name":"HELLO","value":"world"},{"name":"TEST","value":"test_that"}],"resources":{},"volu
meMounts":[{"name":"test-vol","mountPath":"/tmp/test"}],"imagePullPolicy":"IfNotPresent"}},{"op":"add","path":"/spec/containers/0/env","value":[{"name":"HELLO","value":"world"}]},{"op":"add","path":"
/spec/containers/0/env/-","value":{"name":"TEST","value":"test_that"}},{"op":"add","path":"/spec/containers/0/volumeMounts/-","value":{"name":"test-vol","mountPath":"/tmp/test"}},{"op":"add","path":"
/spec/volumes/-","value":{"name":"test-vol","configMap":{"name":"test-config"}}},{"op":"add","path":"/metadata/annotations/injector.tumblr.com~1status","value":"injected"}]

Happens always. All the yamls are from the example in the docs folder.

Version Deets

Kubernetes Version: 1.15.3
k8s-sidecar-injector Top of tree

byxorna commented 5 years ago

Hey @kgunjikar, thanks for the issue! This should be fixed by https://github.com/tumblr/k8s-sidecar-injector/pull/30. Mind taking a peek?

kgunjikar commented 5 years ago

Thanks for the response. if you could please add a sample config, it would be great.

byxorna commented 5 years ago

Thanks for the response. if you could please add a sample config, it would be great.

https://github.com/tumblr/k8s-sidecar-injector/pull/30/files#diff-67e99b25c650f7fe0288309c725f40ad is used by the unit tests to assert that the serviceAccountName is overwritten (https://github.com/tumblr/k8s-sidecar-injector/pull/30/files#diff-31dfa6243f3cee9b9b95fdc19408f98b is the generated response). Just serviceAccountName should be enough to make it work. Your log output makes me think either it isnt using the PR code 🤔

kgunjikar commented 5 years ago

Hmm, maybe I'm missing some config wrt service account. Will get back

kgunjikar commented 5 years ago

I can see the serviceAccount but it doesn't mount in the container. There is nothing in the /var/run/secrets

Codewise, do we need to add a specific volume-mount for the serviceAccount ?

I1024 02:32:21.459356       1 webhook.go:493] AdmissionResponse: patch=[{"op":"add","path":"/spec/containers/-","value":{"name":"sidecar-wiper","image":"diamanti/wiper:0.2","ports":[{"containerPort":80}],"env":[{"name":"ENV_IN_SIDECAR","value":"test-in-sidecar"},{"name":"HELLO","value":"world"},{"name":"TEST","value":"test_that"}],"resources":{},"volumeMounts":[{"name":"test-vol","mountPath":"/tmp/test"}],"imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}}},{"op":"add","path":"/spec/containers/0/env","value":[{"name":"HELLO","value":"world"}]},{"op":"add","path":"/spec/containers/0/env/-","value":{"name":"TEST","value":"test_that"}},{"op":"add","path":"/spec/containers/0/volumeMounts/-","value":{"name":"test-vol","mountPath":"/tmp/test"}},{"op":"add","path":"/spec/volumes/-","value":{"name":"test-vol","configMap":{"name":"test-config"}}},**{"op":"replace","path":"/spec/serviceAccountName","value":"default"},**{"op":"add","path":"/metadata/annotations/injector.tumblr.com~1status","value":"injected"}]
I1024 02:32:21.459650       1 webhook.go:571] Ready to write reponse ...
172.16.190.12 - - [24/Oct/2019:02:32:21 +0000] "POST /mutate?timeout=30s HTTP/2.0" 200 1389 "" "kube-apiserver-admission"
172.16.190.14 - - [24/Oct/2019:02:32:27 +0000] "GET /health HTTP/2.0" 200 12 "" "kube-probe/1.14"
-bash-4.2$ kubectl exec -it debian-debug -c sidecar-wiper /bin/bash
[root@debian-debug /]# ls
anaconda-post.log  bin  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
[root@debian-debug /]# cd /var/run/
[root@debian-debug run]# ls
console  cryptsetup  faillock  lock  log  secrets  sepermit  setrans  systemd  user  utmp
[root@debian-debug run]# cd secrets/
[root@debian-debug secrets]# ls
[root@debian-debug secrets]# exit

byxorna commented 5 years ago

@kgunjikar that seems like the correct config. The kubernetes Service account controller should handle creating the volumemount when we attach the service account to the pod. Can you show the pod's full yaml after injection? This can show whether the pod actually has mounts and SAs configured.

There is an outside possibility that the version of k8s you are running is not rerunning the Service account controller after we mutate the pod, so the SA volumes do not get added to the pod when we inject the serviceAccountName field. There was a bug that was supposedly fixed in 1.15 but I have not verified it myself.

kgunjikar commented 5 years ago

My apologies, it was 1.14.3 . With 1.15.3 it works. Thanks for the help

byxorna commented 5 years ago

@kgunjikar thats great, I am glad you got it sorted out! 😄

tumblr / k8s-sidecar-injector

ServiceAccount Token missing in sidecar #33

Version Deets