search

open-iscsi / targetd

Remote configuration of a LIO-based storage appliance
GNU General Public License v3.0
71 stars 28 forks source link

Support ZFS datasets for fs_pool beside btrfs #54

Closed dsonck92 closed 4 years ago

dsonck92 commented 4 years ago

I was recently looking into persistent volume provisioning for kubernetes and came across the iSCSI provisioner which uses targetd as backend which recently got ZFS support for iSCSI targets. After installing this on my storage host (and patching setup.py with #53) I managed to get it to work which is great.

I realized soon after configuring the ZFS options that targetd also supports NFS exports. But as I looked deeper, this is tied to btrfs. It might be useful to provide the same kind of solution to export ZFS datasets over NFS.

I'm actually willing to see if I can add support for this in a similar way as ZFS zvols are supported now. Although I think probing the target directory for its filesystem instead of providing a configuration option like "zfs_fs_pools" might be preferred.

@triluch considering you added ZFS zvol support, did you consider this already? If you did, any reasons for not (yet) including it?

triluch commented 4 years ago

@dsonck92 I didn't really consider it because:

  1. Getting ZFS as NFS is whole other story than exporting iscsi (another, pretty much separate part of targetd), and that was also what I needed at the time (also for kubernetes and iscsi provisioner). Since there was feature request for that in targetd project I made PR with my changes.
  2. I have noticed there are some ZFS (as NFS) providers for kubernetes (like https://github.com/gentics/kubernetes-zfs-provisioner), so for that usecase there is something already out there. Also I don't think there is a provisioner which uses targetd as source for NFS volumes (iscsi-provisioner just does iscsi, right?).

Of course usecases are not limited to kubernetes providers, so it may also be useful to allow ZFS to be used as NFS exports. But there is probably less demand for this and at this moment. I don't think that feature is worth the work involved. However if you would like to try I can take a look for sure :)

dsonck92 commented 4 years ago

Yes I noticed that it was more or less a separate part inside nfs.py and fs.py. The kubernetes-zfs-provisioner looks interesting however I cannot currently tell if it's possible to have it externally as targetd currently allows. I keep my storage separate from kubernetes (as a good practice) and I intend to keep it that way.

Indeed the iscsi-provisioner currently only handles iscsi but I imagine it should not be too difficult to actually let it create NFS shares based on the configuration it receives from the StorageClass. In my case I need to use NFS for some WriteMany volumes I also want to access externally.

On the other hand, adapting the iscsi-provisioner to an nfs-provisioner with targetd backend should not be too difficult either and might be cleaner in a sense.

I will look into how much effort it takes to add it, Python (and go in case of iscsi-provisioner) is not unknown to me and from an initial look, NFS seems to be written with multiple backends in mind, just btrfs being the only implementation of it.

-- EDIT: that comment and close is way too close

dsonck92 commented 4 years ago

I've started on this over at https://github.com/dsonck92/targetd/tree/zfs-nfs-exports and currently at a stage where fs is forwarding information to backends.btrfs in a filesystem agnostic way. The special subvolumes for snapshots and subvolumes don't work on zfs and as such have been included in the backend. Instead, the fs package relies solely on uuids and resolves the actual names behind them which are then forwarded to the backends.

This results in the btrfs backend being able to map them to pool/targetd_fs/subvol and pool/targetd_ss/snapshot, while ZFS can resolve this to pool/subvol and pool/subvol@snapshot, with the ability to map readonly views from pool/subvol/.zfs/snapshots/snapshot

dsonck92 commented 4 years ago

Meanwhile after implementing it I find https://github.com/democratic-csi/democratic-csi which actually handles remote zfs on linux for both iSCSI and NFS for kubernetes, which would exactly fit my use case. However, I prefer the targetd version as it's eventually less kubernetes centric and I would just need to monitor the targetd daemon instead of some pod that needs some elevated ssh access to setup pools and exports.

In addition, it seems to leverage the native ZFS NFS export function which could clash with my handwritten /etc/exports and had some caveats last time I used it so I still consider this issue and my pull request not obsolete.