search

uswitch / kiam

Integrate AWS IAM with Kubernetes
Apache License 2.0
1.15k stars 238 forks source link

Errors "error finding role for pod" in logs for CronJob's pods #266

Open axozoid opened 5 years ago

axozoid commented 5 years ago

Hi guys,

We've got a few apps running as CronJobs. As soon as pods created by CronJobs we can see a lot of errors like this: ... {"level":"info","msg":"configuring iptables","time":"2019-07-10T06:33:01Z"} {"level":"info","msg":"started prometheus metric listener 0.0.0.0:9620","time":"2019-07-10T06:33:01Z"} {"level":"info","msg":"listening :8181","time":"2019-07-10T06:33:01Z"} {"level":"warning","msg":"error finding role for pod: rpc error: code = Unknown desc = pod not found","pod.ip":"100.96.56.131","time":"2019-07-11T03:07:21Z"} {"level":"warning","msg":"error finding role for pod: rpc error: code = Unknown desc = pod not found","pod.ip":"100.96.56.131","time":"2019-07-11T03:07:21Z"} {"level":"warning","msg":"error finding role for pod: rpc error: code = Unknown desc = pod not found","pod.ip":"100.96.56.131","time":"2019-07-11T03:07:21Z"} {"level":"warning","msg":"error finding role for pod: rpc error: code = Unknown desc = pod not found","pod.ip":"100.96.56.131","time":"2019-07-11T03:07:21Z"} {"level":"warning","msg":"error finding role for pod: rpc error: code = Unknown desc = pod not found","pod.ip":"100.96.56.131","time":"2019-07-11T03:07:21Z"} {"level":"warning","msg":"error finding role for pod: rpc error: code = Unknown desc = pod not found","pod.ip":"100.96.56.131","time":"2019-07-11T03:07:21Z"} {"level":"warning","msg":"error finding role for pod: rpc error: code = Unknown desc = pod not found","pod.ip":"100.96.56.137","time":"2019-07-11T04:06:17Z"} {"level":"warning","msg":"error finding role for pod: rpc error: code = Unknown desc = pod not found","pod.ip":"100.96.56.137","time":"2019-07-11T04:06:17Z"} {"level":"warning","msg":"error finding role for pod: rpc error: code = Unknown desc = pod not found","pod.ip":"100.96.56.137","time":"2019-07-11T04:06:17Z"} {"level":"warning","msg":"error finding role for pod: rpc error: code = Unknown desc = pod not found","pod.ip":"100.96.56.137","time":"2019-07-11T04:06:17Z"} {"level":"warning","msg":"error finding role for pod: rpc error: code = Unknown desc = pod not found","pod.ip":"100.96.56.137","time":"2019-07-11T04:06:17Z"} ....

Checking the logs on the pods created by CronJobs (our application) I see that the job completed without any errors (the app was able to use AWS credentials).

Could someone shed some light on this behaviour, please?

edify42 commented 5 years ago

possibly related https://github.com/uswitch/kiam/issues/47