NFS Export with xfs filesystem

[cduchenoy]

Hi,

With this configuration

$ linstor rg lp nfs_group -p
+-------------------------+
| Key             | Value |
|=========================|
| FileSystem/Type | xfs   |
+-------------------------+

$ linstor-gateway nfs create shares 10.20.117.254/32 10G --allowed-ips=10.20.117.0/24 --resource-group=nfs_group
Created export 'shares' at 10.20.117.254:/srv/gateway-exports/shares

linstor-gateway generate a drbd-reactor config file for an ext4 filesystem:

$ cat /etc/drbd-reactor.d/linstor-gateway-nfs-shares.toml | grep ext4
        "ocf:heartbeat:Filesystem fs_cluster_private device=/dev/drbd/by-res/shares/0 directory=/srv/ha/internal/shares fstype=ext4 run_fsck=no",
        "ocf:heartbeat:Filesystem fs_1 device=/dev/drbd/by-res/shares/1 directory=/srv/gateway-exports/shares fstype=ext4 run_fsck=no",

with FileSystem/Type ext4 setting all are fine

[chrboe]

Hi, thanks for the report!

I agree that this is not very intuitive, but this is actually the intended behavior.

Because we set the all_squash NFS option, the root of the file system must be owned by nobody:nobody (not root:root, which is the default). Unfortunately, we cannot just do a chown nobody:nobody / after the mkfs, because LINSTOR actually makes the mkfs call for us (on another machine). For ext4, we can accomplish this at mkfs time using the root_owner flag (see here, and its usage). Such an option is unfortunately not available for mkfs.xfs.

This is why, currently, only ext4 is supported (and I could have sworn that this fact is documented somewhere...)

Since linstor-gateway explicitly sets the FileSystem/Type property on the volume, it overrides the value you set on the resource group.

I'll close this issue, but I'll try to figure out a way to more clearly hint the user towards this fact.

Of course, if you have an idea on how to implement the same logic for xfs, we are all ears :slightly_smiling_face:

[cduchenoy]

Hi, thanks for the technical explanation!

I followed the official doc at https://linbit.com/drbd-user-guide/linstor-guide-1_0-en/#s-linstor-gateway-create-nfs

Here are the points that I think need improvement:

• only ext4 FS
• only one nfs export
• systemd minimal version 243 (ExecCondition used)
• mkdir -p /srv/ha/internal
• mkdir -p /srv/gateway-exports

Moreover, if the linstor API is secured by an ssl certificate, we have not found a way to pass the client certificate in the linstor-gateway configuration...

I will report an issue on this point with a hack (haproxy)

Instead of closing bug reports, could you classify them as improvements? same as #9

[chrboe]

I fully agree with the first two points. Especially the fact that you can only create one NFS export is a big pain point IMO, and we are working with the LINSTOR team to find solutions to improve this.

Regarding the systemd version: this is an upstream "issue" in drbd-reactor, but the ExecCondition warning can safely be ignored; it is not required for its operation. The oldest systemd version where I have personally successfully run linstor-gateway is 219.

I'm not exactly sure what you mean by the last two points... Could you elaborate? LINSTOR should automatically create these directories.

That is an interesting point regarding the SSL certificate; I will play around with that. Not many people use that LINSTOR feature, so it has never come up.

Sorry if closing the issue came across as dismissive. We are definitely working on improving these areas; I just don't like having many issues open for things that are not currently planned.

Yesterday I actually thought of another route we could take in order to possibly support XFS NFS shares, so I guess we can reopen this issue as a reminder.

[cduchenoy]

We use the linstor-gateway go binaries and we have to manually create the directories for ocf* service to start

[RumenBlack84]

Do these same limitations apply to zfs? I was really hoping to use the linstor gateway to just export a zfs dataset that is being replicated using drbd.