Bricks are not mounted after Gluster node restart

[ghost]

Hello, I have an issue with GlusterFS (Red Hat Gluster Server) cluster running outside of Kubernetes, but managed by an Heketi instance running as a pod.

My configuration is:

• kubernetes cluster v.1.11.3
• RHGS v.3.12
• Heketi v.8.0

My RHGS cluster is running outside of the K8s cluster, in a dedicated environment. Heketi is instead running as a Pod in the K8s cluster.

After restarting one (actually, all) the GlusterFS nodes, no volume cames up, because the /var/lib/heketi/fstabmounts have not been performed automatically. This leads me to the question: who is supposed to configure the GlusterFS nodes to perform those mounts automatically? I took for granted that it was part of the Heketi/GlusterFS setup, but it seems not. Or it is, and I have to look to some specific service which has not been started correctly?

Thanks!

[phlogistonjohn]

So generally, when heketi is managing dedicated gluster nodes rather than gluster-in-containers (from here called an "independent" cluster) heketi must be configured to update /etc/fstab rather than /var/lib/heketi/fstab. Only the container based systems have scripts that operate on /var/lib/heketi/fstab. However, if you chose to use /var/lib/heketi/fstab for a reason, like you don't want heketi changing /etc/fstab you could create your own "init script" to run 'mount -a --fstab /var/lib/heketi/fstab'. You'd need to have this run before glusterd starts.

[ghost]

Ok, fine for me to create my own init script. Thanks for the quick answer!

Maybe not the right place to ask, but let me try :-) From your answer, I can understand that it's possible to choose which file Heketi will write its own mount: may you please address me where this configuration can be done? Would it be possible to do it also when Heketi runs as a Pod?

[phlogistonjohn]

I was assuming that you installed using gk-deploy. If not let me know. When you use it it comes with a template config file for heketi. If you modify this file it will take your modifications. However, if you do not specify "fstab" in the config anywhere it should automatically use /etc/fstab when you use the ssh executor. How heketi itself is run, container or not, doesn't matter. It's fully controlled by the config file and the environment vars.

[ghost]

No, I'm not using gk-deploy. I installed RHGS using yum, then I followed the guide that can be found here: https://github.com/heketi/heketi/blob/46a8eaa3bfbc488b35ebaa27ff7ca12a594fe549/docs/admin/install-kubernetes.md, skipping the GlusterFS installation part. I guess that the file that Heketi is using to write its own mounts is part of the Heketi configuration, isn't it? Can you tell me were can I find it?

Thanks again for your support!

[ghost]

I think I've got it:

    Environment:
      HEKETI_DB_PATH:         /var/lib/heketi/heketi.db
      HEKETI_FSTAB:           /var/lib/heketi/fstab
      HEKETI_SNAPSHOT_LIMIT:  14

I don't know if I can close this issue by myself, or if you should do.

Thanks again!