search

gluster / gluster-kubernetes

GlusterFS Native Storage Service for Kubernetes
Apache License 2.0
875 stars 389 forks source link

Unable to create gluster cluster due to unwritable host /usr/lib #537

Closed fdr2 closed 5 years ago

fdr2 commented 5 years ago

I am using an existing kubernetes cluster. Added a topology.json that matches my layout.

When I create a new gluster/heketi cluster I receive the error message "Error: failed to start container "glusterfs": Error response from daemon: error while creating mount source path '/usr/lib/modules': mkdir /usr/lib/modules: read-only file system" in my pods.

According to the template this path should be mounted read-only, but it does not exist on my nodes.

But when I attempt to create the path manually, the device this is mounted on is not writable (this folder also does not exist, so it would be empty even if it was a writable filesystem).

$ sudo mkdir /usr/lib/modules
mkdir: cannot create directory ‘/usr/lib/modules’: Read-only file system

Is it possible to move this path to a different host location? I tried to discern its purpose from the image's source, but couldn't find its relevance.

nixpanic commented 5 years ago

The /usr/lib/modules directory is needed so that the processes in the glusterfs pod can load kernel modules. Could you check where your host has the modules available? A command like this would show the full path to the fuse module:

$ modinfo --filename fuse
/lib/modules/4.18.18-300.fc29.x86_64/kernel/fs/fuse/fuse.ko.xz

What distribution are you using?

fdr2 commented 5 years ago

/lib/modules/4.4.111+/kernel/fs/fuse/fuse.ko Container-Optimized OS

nixpanic commented 5 years ago

In that case, you should be able to change the path /usr/lib/modules to /lib/modules in deploy/kube-templates/glusterfs-daemonset.yaml.

This should work on most distributions as /usr/lib/modules is the new location for many, but they do provide /lib as a symlink to /usr/lib.

fdr2 commented 5 years ago

Thank you very much for your help, Niels