[enhancement/suggestion] O3X/ZoL pool compatibility

[RJVB]

This already came up on the forum but today's run-in with an O3X pool that imports but appears to be empty under ZoL (0.7.3) inspired me to file an issue about it here.

Upgraded or newly created pools with default settings are no longer compatible between ZFS on Linux and OpenZFSonOSX because each version now has at least 1 attribute that the other doesn't support:

• O3X has com.datto:encryption (I admit I haven't yet verified its effect on Linux)
• ZoL has org.zfsonlinux:userobj_accounting and org.zfsonlinux:large_dnode which both make pool read-only compatible at most.

I don't doubt that ZFS doesn't aim to be an exchange file system. Yet, OS X/macOS and Linux are the 2 most widely used desktop OSes for which ZFS implementations exist and they do not currently share another native (non-FUSE) Unix file system (beyond NFS). I think it is thus important aim to maintain compatibility. I use this on my Mac to access files on backups of my Linux rigs which all use ZFS for everything but the boot partition.

Example: it might (would probably be?) acceptable for most desktop users if changing files on a ZoL pool with userobj_accounting reset that information for those files because there's no actual implementation for it in O3X. (Users for whom this is not the case can keep the fact in mind if it's documented properly.)

[lundman]

ZOL has com.datto:encryption in master, just not in a tagged release just yet. It is their feature.

As for org.zfsonlinux:userobj_accounting and org.zfsonlinux:large_dnode - the usual procedure is for ZOL to do PR to openzfs-repo (and they have), and once that is accepted (they have not been), we will pull it down to OSX. The top of the hierarchy is openzfs-repo.

But, the zpool features were designed such that you should create a pool that does not use the features known to not work other platforms (zpool create -d). Personally, I'm not sure the feature system is "great" as is, since everything is "on" by default, so you'll always end up in compatibility problems until all platforms are identical.

[RJVB]

the usual procedure is for ZOL to do PR to openzfs-repo (and they have), and once that is accepted (they have not been)

I know, I thought it would hurt to have an issue up about this as a reminder not to forget about those PRs (and possibly stir things up from time to time; I don't think that'd be up to an outside like me).

Yes, the feature system could support "latent" features that you need to activate at the risk of getting an incompatible pool.Or to allow turning features back off. And couldn't there be some kind of compatibility mode for features where this makes sense, a mode in which certain implementations preserve existing information but don't use or update it? That wouldn't work for encryption of course (if that goes beyond file contents; if not you can just present encrypted files as random noise data) but it would for accounting info and possibly for certain settings of the large dnode feature.

Let's hope those PRs get accepted quickly. Because in spite with what I said in my OP, you will probably feel a bit stupid when you realise that this reliable backup system you thought you had turns out not to mount on another platform just when you need it.

(Hate to admit it, but I do have a few backups I keep as relics that were made with the wrong reliable technology - a tape cartridge, some exotic UFS, a ZIP disk, ... =] )

[ilovezfs]

Until upstream implements something, -d is the way to handle this.

[RJVB]

Until upstream implements something, -d is the way to handle this.

What -d?

[ilovezfs]

But, the zpool features were designed such that you should create a pool that does not use the features known to not work other platforms (zpool create -d)

[ilovezfs]

https://github.com/illumos/illumos-gate/blob/b0e142e57dfda6f02bf1fa66973c610f5fefe8cb/usr/src/man/man1m/zpool.1m#L850-L859

[athompso]

There's also the issue of people (like me) who run the "upgrade" command on Linux without remembering that this disk needs to be accessible (r/w) on MacOS. And then blow away their Linux system :-(. Yes, I painted myself into a corner, but I would like to be able to escape somehow - ignoring/disabling that feature bit on import, possibly?

[lundman]

If you are able to compile and run own built, I can make the userobj and large_dnode PRs available in a branch for you. But yeah, in hind-sight obviously don't run zpool upgrade. Which is actually worth remember in general, unless it has a specific new feature you really need, upgrading is mostly no point.

[RJVB]

unless it has a specific new feature you really need, upgrading is mostly no point.

This also came up on the forum for features now available in ZoL but not yet in OpenZFS (so on Mac too) including the fact that new features are enabled by default when you create a new pool. (Those were userobj_accounting and large_dnode IIRC, so yay if these made it to O3X.)

It could be useful and convenient to have some way (build option?) to create new pools in compatibility ("not upgraded") mode where features not available on all (major) platforms aren't activated/enabled by default (rather than having to disable them explicitly).

[ilovezfs]

@RJVB please take that up with upstream.

[lundman]

I think ZOL changed the large-dnode, to now have a dataset property, and its value is "legacy" and you have to additionally turn it on? At least, I assume that's why they added it. As for userobjaccounting, I really think they should turn the feature on only after a user has quota actually set, instead of always on. But there might be some details of it I am unsure of.

[RJVB]

Yes, I think the large_dnode feature isn't a problem until you used it once on a dataset.

[athompso]

Meanwhile, @lundman, I appreciate the offer but I'm not that desperate yet.

[RJVB]

I see I misinterpreted something earlier (in my defense, I'm on a family trip) :-/ Will the userobj and large_dnode PRs require additional adaptation for O3X? If so, it might indeed make sense to make them available in a (temporary) branch (?)

[ilovezfs]

They're not coming in unless they're merged in https://github.com/openzfs/openzfs/ so there's little point in PRs here at this time.