posix permissions handler does not function as desired when using NFS as backend FS

[mGurusamy]

posix permissions handler while exposing existing pvc

while trying to expose the existing pvc as smbshare, initContainer is giving error saying that posix permissions handler OSError: [Errno 95] Operation not supported. Hence the SmbShare deployment is not up and running.

logs from init container ensure-share-paths

2023-05-25 20:51:48,459: INFO: Using initializing posix permissions handler
Traceback (most recent call last):
  File "/usr/bin/samba-container", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/sambacc/commands/main.py", line 231, in main
    cfunc(CommandContext(cli))
  File "/usr/lib/python3.11/site-packages/sambacc/commands/initialize.py", line 91, in ensure_share_paths
    perms_handler(share.permissions_config(), path).update()
  File "/usr/lib/python3.11/site-packages/sambacc/permissions.py", line 143, in update
    if self.status_ok():
       ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/sambacc/permissions.py", line 136, in status_ok
    sval = self._get_status()
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/sambacc/permissions.py", line 152, in _get_status
    value = xattr.get(path, self._xattr, nofollow=True)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/sambacc/_xattr.py", line 58, in get
    return xattr.get(item, name, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 95] Operation not supported

Thought this is something to do with smb driver version while mounting. Hence updated the corresponding config map's globals.globals.options as "version": "2.0"

 "globals": {
        "globals": {
          "options": {
            "disable spoolss": "yes",
            "dos charset": "ascii",
            "fileid:algorithm": "fsid", 
            "load printers": "no",
            "printcap name": "/dev/null",
            "printing": "bsd",
            "smb ports": "445",
            "vfs objects": "fileid",
            "version": "2.0"
          }
        }
      },

and restarted the deployment. But still the deployment's initContainer(ensure-share-paths) is giving the same Error. please let me know if any one come across this problem and solved this issue. Thank you

[phlogistonjohn]

Hi there, We've seen this issue pop up a couple of times before and in those cases the underlying pv file system did not support xattrs. I think in those cases the underlying fs was also NFS? Are you also trying to use NFS? If not, what is the underlying file system type on the PV in question?

References: #306, #294

Thought this is something to do with smb driver version while mounting. Hence updated the corresponding config map's globals.globals.options as "version": "2.0"

I don't think this will do anything. AFAIK, "version" is not a field samba (smb.conf) understands. On top of that the xattr calls are being performed by the sambacc library which is a wrapper that runs before launching one of the samba servers. This is configurable at the sambacc layer but not at the samba-operator today.

[mGurusamy]

Thank you @phlogistonjohn for your reply. Really appreciate your time. Yes the underlying volume backend file system type is NFS4. we wanted to expose those volumes as smbshare without disturbing the current pod setup.

[phlogistonjohn]

Thanks for letting us know. It comes as a bit of a surprise to me but it appears to be common-ish use case to try and nest (Samba) SMB on top of NFS. There are issues with that sort of nesting and it would be better to use a non-NFS file system [1] but I will leave this issue open for two sub-tasks:

1. Adding an option to disable sambacc permissions handling
2. Adding documentation for the above change and an explicit discussion of the SMB on NFS topic

[1] Local file systems like ext4, xfs, or for clustered network file systems - CephFS (because we test with that)

[tomaszov]

Dear @phlogistonjohn I'm also facing this issue. I might misunderstand your reply, but do I understand correctly that option 1 is possible in some way? Could you please help me in how to achieve this? Regarding the surprising use case, for us it is simply that we have several pods with RWX volumes, that are actually home directories of our users. We are looking for a native and easy to handle way for them to copy files there from their Windows workstations.

[phlogistonjohn]

Hi @tomaszov option 1 would require a code change in the operator IIRC. The containers and samba can do this but the operator unconditionally sets the sambacc options to enable the permissions handler.

If you or any of the others running into this issue know a little Go I'd be thrilled to see contributions in this area. Otherwise, I hope to find the time to get to this eventually.

As for the "surprise" - the use case makes sense but I didn't expect folks to stack one network file system (SMB) on another (NFS). I was always testing with a clustered system (Rook/Ceph) as the underlying storage. Plus, I am aware of issues with that stacking (even outside of k8s) from years of reading the Samba mailing list. But I'm not that surprised either. ;-)

[tomaszov]

Thank you @phlogistonjohn for your answer! Unfortunately personally I have zero experience with Go, nor my team mates have any. A bit worries me that you are mentioning issues with this solution anyway. May I ask you to drop me some links on it? (Googling just gave me comparisons of the two) It might be that after all this way would not work for us anyway, as we need a stable solution. We moved away from Rook/Ceph but other alternatives use NFS for RWX mostly AFAIK.

[phlogistonjohn]

https://lists.samba.org/archive/samba/2023-March/244535.html

https://linux.die.net/man/5/smb.conf (search for posix locking section)

https://lists.samba.org/archive/samba/2013-June/173757.html

https://lists.samba.org/archive/samba/2009-May/148403.html

[pfaelzerchen]

The longhorn Storage Provider mounts its RWX volumes by default with nfs version 4.1. So that is where my use case comes from. As my Samba use case is somewhat small, I would like to keep longhorn. But as the PVCs need to be mounted to several other pods (I also expose sshfs) RWX is the only option.

For anyone who can live with read only shares, you can set share.permissions.method to "none" in the config map which will keep sambacc from trying to set xattrs. Then NFS v4.1 will work.

[gelato]

Yes, I also came across this issue trying to run this operator ontop of Longhorn's RWX volume. It is very disappointing that I can't do that, Longhorn is a silver bullet solution for me...

[pfaelzerchen]

I think I see the point why SMB on NFS isn't a good idea. The longhorn NFS mounts weren't really stable after all (connection timeouts causing parts of the cluster to go down). So I switched to rook/ceph mentioned above. More troublesome to deploy, but more stable afterwards. And the samba-operator works as expected.