nfsmount.conf nolock option ignored when using democratic-csi nfs support

bplein commented 3 months ago

nfsmount.conf nolock option ignored when using democratic-csi zfs-generic-nfs support:

Warning  FailedMount             11s   kubelet                  MountVolume.MountDevice failed for volume "pvc-4f86d14f-16d5-49b8-ba51-84319e159b50" : rpc error: code = Internal desc = {"code":32,"stdout":"","stderr":"/usr/local/bin/mount: illegal option -- o\nmount.nfs: rpc.statd is not running but is required for remote locking.\nmount.nfs: Either use '-o nolock' to keep locks local, or start statd.\n","timeout":false}

Description

I have installed and configured democratic-csi generic ZFS NFS support. When attempting to create RWX volumes, I got the above error. I found this past issue #6582 that led to a "fix" by including /etc/nfsmount.conf with the nolock option. I see that file is present in Talos v1.7.4.

Since I'm still getting the errors, that must be ignored (could democratic-csi be bypassing it?).

I then created a test storageclass that includes the lock option:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: zfs-nfs-nolock
provisioner: org.democratic-csi.nfs
mountOptions:
- noatime
- nfsvers=3
- nolock
parameters:
  csi.storage.k8s.io/controller-expand-secret-name: controller-expand-secret-zfs-nfs-nvme-zfs-nfs-democratic-csi
  csi.storage.k8s.io/controller-expand-secret-namespace: democratic-csi
  csi.storage.k8s.io/controller-publish-secret-name: controller-publish-secret-zfs-nfs-nvme-zfs-nfs-democratic-csi
  csi.storage.k8s.io/controller-publish-secret-namespace: democratic-csi
  csi.storage.k8s.io/node-publish-secret-name: node-publish-secret-zfs-nfs-nvme-zfs-nfs-democratic-csi
  csi.storage.k8s.io/node-publish-secret-namespace: democratic-csi
  csi.storage.k8s.io/node-stage-secret-name: node-stage-secret-zfs-nfs-nvme-zfs-nfs-democratic-csi
  csi.storage.k8s.io/node-stage-secret-namespace: democratic-csi
  csi.storage.k8s.io/provisioner-secret-name: provisioner-secret-zfs-nfs-nvme-zfs-nfs-democratic-csi
  csi.storage.k8s.io/provisioner-secret-namespace: democratic-csi
  fsType: nfs
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true

This allows the PVC to be created correctly.

Logs

Events:
  Type     Reason                  Age                From                     Message
  ----     ------                  ----               ----                     -------
  Warning  FailedScheduling        46s                default-scheduler        0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.
  Normal   Scheduled               45s                default-scheduler        Successfully assigned default/busybox-rwx-deployment-csi-78f8d66d8f-qfxc9 to rk1-1
  Normal   SuccessfulAttachVolume  45s                attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-2c936b9c-5d6a-461e-9b64-7897f0c0dd48"
  Warning  FailedMount             10s (x3 over 33s)  kubelet                  MountVolume.MountDevice failed for volume "pvc-2c936b9c-5d6a-461e-9b64-7897f0c0dd48" : rpc error: code = Internal desc = {"code":32,"stdout":"","stderr":"/usr/local/bin/mount: illegal option -- o\nmount.nfs: rpc.statd is not running but is required for remote locking.\nmount.nfs: Either use '-o nolock' to keep locks local, or start statd.\n","timeout":false}

Environment

Talos version: Tag: v1.7.4 SHA: cb3a8308 Go version: go1.22.3 OS/Arch: darwin/amd64
Kubernetes version: [Client Version: v1.20.2 Server Version: v1.30.1]
Platform: TuringPi RK1

bplein commented 3 months ago

=========== One additional comment: The nolock option is a workaround. Talos should support running rpc.statd (per the original error message) in order to support other locking options. Is it feasible to add this support (for rpc.statd?)

zviratko commented 1 month ago

I also think Talos should support running rpc.statd. NFS might be an unwanted child, but it is there to stay. Locking support is essential in some apps....

siderolabs / talos