search

tumblr / k8s-sidecar-injector

Kubernetes sidecar injection service
Apache License 2.0
345 stars 75 forks source link

sidecar injected though no configmap created, therefore init does not come up #58

Open peppe77 opened 3 years ago

peppe77 commented 3 years ago

K8S: 1.18 sidecar: 0.1.7

the side-car gets injected though the required configmap is not created, therefore the init does not come up. this only happens in one namespace. similar setup works in 2 other namespaces. some suggestions on which additional logs to enable and check? problem is still happening, so should be fairly easy to get more logs. thanks

MountVolume.SetUp failed for volume "vault-agent-init-config" : configmap "vault-agent-init-config" not found

sidecar-injector-6b9977dfdf-fwk75 sidecar-injector E0616 19:10:19.980139       1 main.go:118] watcher got error, try to restart watcher: watcher channel has closed
sidecar-injector-6b9977dfdf-fwk75 sidecar-injector I0616 19:10:19.980145       1 main.go:113] launching watcher for ConfigMaps
sidecar-injector-6b9977dfdf-fwk75 sidecar-injector 10.50.5.106 - - [16/Jun/2021:19:10:22 +0000] "GET /health HTTP/2.0" 200 12 "" "kube-probe/1.18"
sidecar-injector-6b9977dfdf-fwk75 sidecar-injector I0616 19:10:22.982614       1 main.go:129] triggering ConfigMap reconciliation
sidecar-injector-6b9977dfdf-fwk75 sidecar-injector I0616 19:10:22.982640       1 watcher.go:151] Fetching ConfigMaps...
sidecar-injector-6b9977dfdf-fwk75 sidecar-injector I0616 19:10:23.028036       1 watcher.go:158] Fetched 1 ConfigMaps
sidecar-injector-6b9977dfdf-fwk75 sidecar-injector I0616 19:10:23.028520       1 watcher.go:179] Loaded InjectionConfig vault-auth from ConfigMap sidecar-injector-default:vault-auth
sidecar-injector-6b9977dfdf-fwk75 sidecar-injector I0616 19:10:23.028897       1 watcher.go:179] Loaded InjectionConfig vault-auth-init from ConfigMap sidecar-injector-default:vault-auth-init
sidecar-injector-6b9977dfdf-fwk75 sidecar-injector I0616 19:10:23.028909       1 watcher.go:164] Found 2 InjectionConfigs in sidecar-injector-default
sidecar-injector-6b9977dfdf-fwk75 sidecar-injector I0616 19:10:23.028914       1 main.go:135] got 2 updated InjectionConfigs from reconciliation
sidecar-injector-6b9977dfdf-fwk75 sidecar-injector I0616 19:10:23.028919       1 main.go:149] updating server with newly loaded configurations (3 loaded from disk, 2 loaded from k8s api)
sidecar-injector-6b9977dfdf-fwk75 sidecar-injector I0616 19:10:23.028925       1 main.go:151] configuration replaced
sidecar-injector-6b9977dfdf-hrmvt sidecar-injector I0616 19:15:08.600191       1 webhook.go:435] AdmissionReview for Kind=/v1, Kind=Pod, Namespace=external-god-connectedvehicle-services Name= () UID=b9f0adb9-96ea-4994-be36-d9cfe10e6cf5 patchOperation=CREATE UserInfo={system:serviceaccount:kube-system:job-controller 503d64d0-aa05-11e9-8bbd-0a71b5a65c66 [system:serviceaccounts system:serviceaccounts:kube-system system:authenticated] map[]}
sidecar-injector-6b9977dfdf-hrmvt sidecar-injector I0616 19:15:08.600224       1 webhook.go:163] Pod / annotation injector.tumblr.com/request=vault-auth-init requesting sidecar config vault-auth-init
sidecar-injector-6b9977dfdf-hrmvt sidecar-injector I0616 19:15:08.600316       1 webhook.go:473] AdmissionResponse: patch=[{"op":"add","path":"/spec/containers/0/volumeMounts","value":[{"name":"secrets","mountPath":"/etc/secrets"}]},{"op":"add","path":"/spec/containers/0/volumeMounts/-","value":{"name":"vault-token","mountPath":"/home/vault"}},{"op":"add","path":"/spec/initContainers","value":[{"name":"vault-agent-auth","image":"harbor.infrastructure.volvo.care/infrastructure/vault:1.2.3","args":["agent","-config=/etc/vault/vault-agent-init-config.hcl"],"env":[{"name":"SKIP_SETCAP","value":"true"}],"resources":{"limits":{"cpu":"150m","memory":"250Mi"},"requests":{"cpu":"50m","memory":"64Mi"}},"volumeMounts":[{"name":"vault-agent-init-config","mountPath":"/etc/vault"},{"name":"vault-auth","readOnly":true,"mountPath":"/var/run/secret"},{"name":"secrets","mountPath":"/etc/secrets"},{"name":"vault-token","mountPath":"/home/vault"}],"securityContext":{"privileged":false,"runAsUser":100,"runAsGroup":1000,"runAsNonRoot":true,"allowPrivilegeEscalation":false}}]},{"op":"add","path":"/spec/volumes","value":[{"name":"vault-auth","secret":{"secretName":"vault-sa-token","items":[{"key":"token","path":"token","mode":292}]}}]},{"op":"add","path":"/spec/volumes/-","value":{"name":"vault-token","emptyDir":{"medium":"Memory"}}},{"op":"add","path":"/spec/volumes/-","value":{"name":"vault-agent-init-config","configMap":{"name":"vault-agent-init-config"}}},{"op":"add","path":"/spec/volumes/-","value":{"name":"secrets","emptyDir":{"medium":"Memory"}}},{"op":"add","path":"/metadata/annotations/injector.tumblr.com~1status","value":"injected"}]
sidecar-injector-6b9977dfdf-hrmvt sidecar-injector I0616 19:15:08.600352       1 webhook.go:551] Ready to write reponse ...
sidecar-injector-6b9977dfdf-hrmvt sidecar-injector 100.107.171.128 - - [16/Jun/2021:19:15:08 +0000] "POST /mutate?timeout=30s HTTP/1.1" 200 2145 "" "kube-apiserver-admission"