search

openzfs / zfs

OpenZFS on Linux and FreeBSD
https://openzfs.github.io/openzfs-docs
Other
10.44k stars 1.73k forks source link

Implement NFS4 ACL support #4966

Open ghfields opened 8 years ago

ghfields commented 8 years ago

The zfs filesystem has NFS4 ACLs built right into it. The use of those ACLs work on other OSs, however, its use has never been ported to Linux. A while ago, issue #170 was created for POSIX and NFS4 acl support. It was closed after POSIX support as added, however NFS4 was not implemented.

NFS4 implementation was held up because the tools to manage NFS4 acls were not available. Now, some distros are using the "nfs4-acl-tools" package for NFS4 acl management now. It uses nfs4_getfacl, nfs4_setfacl, and nfs4_editfacl.

In followup discussions in issue #170, @behlendorf stated:

I haven't looked in to nfs4 acls for a few years but it looks like they're making really good progress with the user space components. Based on a cursory reading of the nfs4-acl-tools source it looks like the expected user/kernel interface is via an xattr named system.nfs4_acl which contains the raw xdr encoded acl.

Getting this working might only require adding xattr handlers for a system.nfs4_acl xattr which translates between the nfs4 acl stored internally by ZFS and the representation of it expected by the utilities. Since NFSv4 is the only consumer of this the kernel doesn't provide any generic functionality we can use we'd need to write the functions to do this encoding/decoding.

On the surface getting this working looks very possible. I think it would be great if a developer wanted to tackle this feature.

So, I created an new issue, separate from the closed original #170 issue to track the progress of NFS4.

gdevenyi commented 7 years ago

Just a detail to add to this here. Python3 seems to have added native support to detect/handle ACL stuff when doing various file manipulations within their core libraries. Fundamentally, this means things like setuptools try to set ACLs on nfs4 mounted shares. Since ZFS doesn't support NFS4 ACL, this fails, which means that a bunch of stuff is randomly broken with python3 installs on NFS4 on ZFS.

It comes up like this strace from a run of setuptools to install onto a NFSv4 mounted ZFS volume:

chmod("/opt/quarantine/pydpiper/2.0/build/lib/python3.4/site-packages/pydpiper-2.0-py3.4.egg", 0644) = 0
listxattr("dist/pydpiper-2.0-py3.4.egg", "system.nfs4_acl\0", 256) = 16
getxattr("dist/pydpiper-2.0-py3.4.egg", "system.nfs4_acl", "\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16\x01\x87\x00\x00\x00\x06OWNER@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x06GROUP@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x09EVERYONE@\x00\x00", 128) = 80
setxattr("/opt/quarantine/pydpiper/2.0/build/lib/python3.4/site-packages/pydpiper-2.0-py3.4.egg", "system.nfs4_acl", "\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16\x01\x87\x00\x00\x00\x06OWNER@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x06GROUP@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x09EVERYONE@\x00\x00", 80, 0) = -1 EIO (Input/output error)
maxximino commented 7 years ago

@gdevenyi I think it should be a separate bug. The NFSv4 server code translates the NFSv4 ACL into a proper posix acl, and just ask the filesystem to store that. See http://lxr.free-electrons.com/source/fs/nfsd/nfs4acl.c?v=4.7#L776 / http://lxr.free-electrons.com/source/fs/nfsd/nfs4acl.c?v=4.7#L102 (assuming you're using the in-kernel NFS server).

gdevenyi commented 7 years ago

@maxximino I am indeed using the in-kernel stuff. Are you suggesting a separate ZFS bug or is this an NFSd bug?

maxximino commented 7 years ago

@gdevenyi separate ZFS bug, if it happens only with ZFS :)

pbhenson commented 7 years ago

Anyone looking at or interested in this? My university used to use zfs under Solaris, then switched to omnios. Unfortunately it appears that omnios is on its way out :(, so I'm taking a look at possible replacements for the future. The rest of our infrastructure runs mostly linux, so zfs under linux sounds attractive. However, our primary zfs use is storage export via NFSv4 and CIFS, so the lack of NFSv4 ACL support is a no-go. It would be great from a feature parity perspective for the linux port to support the native zfs acl :).

Thanks...

twerschlein commented 7 years ago

Same situation here (also a University). We went the Solaris -> OmniOS -> FreeBSD path. Out of the box AD integration with Linux (via sssd, including autofs) is far easier than with FreeBSD. Native ZFS ACLs on Linux would be great. Thanks everyone who is working on zfsonlinux!

ksteinb commented 7 years ago

We (Faculty of Physics, Ludwig Maximilians University) would also be highly interested in NFS4 ACL's, We run large storage under Nexenta and are moving currently to ZOL. Loosing the Rich ACL's is a pain.

b-a-t commented 6 years ago

Well, FreeBSD has both ZFS and NFSv4 ACLs, so could be alternative to Solaris based installations.

pbhenson commented 6 years ago

True; but then we'd have to build up from scratch infrastructure for deploying/managing FreeBSD, rather than migrating ZFS services onto our existing Linux infrastructure. If you don't have any existing OS commitment, I'd agree FreeBSD would be an excellent choice, but it would be much more efficient for us to have NFSv4 ACLs on the Linux port. Fortunately at least for now OmniOS seems to have picked up pretty reliable community support so a short-term replacement isn't as critical as it was looking to be.

b-a-t commented 6 years ago

Don't take it as shameless plug, but did you consider to try FreeNAS or, on a commercial grade, TrueNAS

P.S> I came across this issue while checking do we need to keep compatibility with ZOL in Samba's vfs_zfsacl. Seems, not yet.

ksteinb commented 6 years ago

FreeBSD's Network Stack is not so well developed as the Linux Stack regarding high throughput. For most applications it is more than good enough, but not for real high bandwitdh deployments (multiple 10 GE )

Also we do have really large ZOL deployments (in the Petabyte range) in High Availability Configurations (with Pacemaker) as well as ZOL Deployments with LizardFS on top. So FreeNAS/TrueNAS is not really a option for us.

As we do have to support both the Linux as well as the Windows and Mac World, the NFS4acl's would be of great help for us.

sbbeachvball commented 6 years ago

There is a UCSB research contigent that is also very interested in NFSv4 acls on ZOL. We absolutely love ZOL and this is the one catching point. We have posix ACLs on some of our filesystems, which forces us to use NFSv3 on clients. Would completely dig switching those to NFSv4 ACLs! Thanks ZOL team.

pbhenson commented 6 years ago

To avoid duplicate effort in case anybody else is looking at this, I'm going to start working on it. There's nothing comitted yet, but work in progress will be at:

https://github.com/pbhenson/zfs/tree/nfsv4_acl

pbhenson commented 6 years ago

I pushed an initial WIP to my fork, right now it just supports reading ACLs, not updating them, and I'm sure there are unsquashed bugs ;), but I'd appreciate any feedback so as I move on to writing ACLs I don't continue any mistakes I've already made :).

ghfields commented 5 years ago

@pbhenson has done great work on the in #7728. His most recent comment is requesting additional critique on the implementation.

I built the code and tried an "Hail Mary" attempt to use it with Samba4 under Ubuntu 18.04.1. I found Ubuntu 18.04.1 does not build with zfsacl vfs compatibility. https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1788776 . Looks like I get to build Samba also.

pbhenson commented 5 years ago

I built the code and tried an "Hail Mary" attempt to use it with Samba4 under Ubuntu 18.04.1. I found Ubuntu 18.04.1 does not build with zfsacl vfs compatibility. https://bugs.launchpad.net/ubuntu/+source/samba/+bug/1788776 . Looks like I get to build Samba also.

Unfortunately, I believe the samba zfs acl module is dependent on the Solaris/illumos acl API, which Linux does not support, so it will likely not work as-is. Either a compatibility shim would need to be developed mapping the illumos acl API calls to the linux nfs4 extended attribute call, or a different samba vfs module which uses the linux nfs4 api directly would be needed. I'm not sure which would be the better approach, perhaps I'll take a look at it once this work is completed. Need to get motivated to work on those tests :).

b-a-t commented 5 years ago

Unfortunately, I believe the samba zfs acl module is dependent on the Solaris/illumos acl API, which Linux does not support, so it will likely not work as-is. Either a compatibility shim would need to be developed mapping the illumos acl API calls to the linux nfs4 extended attribute call, or a different samba vfs module which uses the linux nfs4 api directly would be needed. I'm not sure which would be the better approach, perhaps I'll take a look at it once this work is completed. Need to get motivated to work on those tests :).

Just recently I had a discussion with my colleague @anodos325 about the way how Samba handles ZFS NFSv4 ACLs now and what could be the possible direction of the development in the future.

At the moment vfs_zfsacl Samba module is using Solaris ACL API, which is quite OS specific and isn't supported by FreeBSD, for example. While ago a compatibility library libsunacl was created by @trasz, which emulates Solaris API through the FreeBSD native API. That seems to work relatively well, but obviously adds performance overhead. So now, when ZoL also got NFSv4 ACLs support the choice is either:

Taking into account the https://github.com/zfsonfreebsd/ZoF initiative I'd love to see some collaboration in the unifying NFSv4 ACLs support on ZFS and, hence it's usage in Samba.

b-a-t commented 5 years ago

there is a freebsd port of the samba VFS module.

That's not the port, but rather compatibility library libsunacl which emulates Solaris NFSv4 API through the FreeBSD ACLs API. That allowed us to keep vfs_zfsacl module intact.

trasz commented 5 years ago

FWIW, I'd be quite surprised if the overhead of libsunacl was actually measurable - it just goes over a table and maps some flags.

pbhenson commented 5 years ago

I'd appreciate it if anyone interested in this could take a look at the current code in the pull request. Perhaps it's a character flaw ;), but I just haven't been able to motivate myself to spend time on the tests given the lack of any reasonable indication this code would be integrated if they were finished...

JoakimZiegler commented 5 years ago

Hey guys, just a quick note that this particular user, at least, would very much appreciate NFS4 ACL support on ZFS. If someone is actively working on it, we might be willing to drop a few bucks on sponsoring this, if that's an option, especially if it's likely to be merged soon (and backported into 0.8.x).

pbhenson commented 5 years ago

I believe the current work in my branch is functionally complete, although it looks like I need to rebase it to get it to apply cleanly to current master. It still needs more code review and developer testing to see what oopsies I might have hiding :). Then it needs some updates to the test suite to cover NFS4 ACL functionality, I'm still a bit stalled on that as I'm not sure exactly how I want to handle it .

JoakimZiegler commented 5 years ago

Well, if there's anything I can do to help, up to and including some financial incentive, please let me know. We are very interested in this, and I don't think we're the only ones.

pbhenson commented 5 years ago

Well, I can't deny spending time on this would become a higher priority if it were a contract job rather than a side project :). However, as I'm not a member of the core zfsonlinux developer team I wouldn't be able to guarantee the code would be committed to the authoritative repo, only that I'd work on whatever changes they request to consider it (which is currently enhancing the test suite to cover the new functionality). Feel free to contact me directly if you'd like to discuss further.

JoakimZiegler commented 5 years ago

Hey, @pbhenson , where can I find your contact info?

pbhenson commented 5 years ago

You can reach me via henson (at) acm (dot) org

bjacke commented 4 years ago

thanks a lot for your efforts to add NFS4 ACL support in openZFS! I linked this issue here on the Samba Wiki's NFS4 ACL overview page. If some Linux distribution would decide to add NFS4 ACL support for ZFS (or even better also the generic RichACL patches) to their kernel, this would make it the distribution of choice for many Samba servers.

benintech commented 7 months ago

No input since 2020. What's the status of this issue in 2024 ? I'm planning to set up a ZFS Samba server, still hesitating between BSD and Linux. Does ZFS/Linux now natively support NFSv4 ACLs ?

gdevenyi commented 7 months ago

Currently draft PR in https://github.com/openzfs/zfs/pull/13186