kahkhang / kube-linode

:whale: Provision a Kubernetes/CoreOS cluster on Linode
MIT License
212 stars 31 forks source link

Rook Agent Pods fail to start #52

Closed kahkhang closed 6 years ago

kahkhang commented 6 years ago

See #51

Error: failed to start container "rook-agent": Error response from daemon:
{"message":"mkdir /usr/libexec/kubernetes: read-only file system"}
Error syncing pod
Back-off restarting failed container

My hunch is that some flag needs to be added to the kubelet.service systemd file (which can be found at https://github.com/kahkhang/kube-linode/blob/master/manifests/container-linux-config.yaml and https://github.com/kahkhang/kube-linode/blob/master/manifests/container-linux-config-worker.yaml) to include --volume-plugin-dir=/etc/kubernetes/volumeplugins, and also probably the kubernetes version needs some bumping up as well to the latest one which supports flex volume plugins.

ZeD3v commented 6 years ago

Same here. In fact yesterday I tried to spin up a cluster of three nodes and it failed to start rook pod.

At first it failed with RBAC error, which I fixed by adding rook-operator serviceaccount to cluster-admin clusterrole.

After that it failed with error saying that it couldn't detect pod name, which was fixed by adding env var POD_NAME to the rook-operator container.

And then I ran into issue described above where apparently it was trying to write to read only mounted volume.

I tried to add volume-plugin argument as KUBELET-ARGS environment variables, tried to add it directly to kubelet-wrapper in kubelet service, but unfortunately with no luck.

Any suggestions welcome.

kahkhang commented 6 years ago

Should be fixed now with #53. cc @displague.