Support for io_uring API?

Harvie commented 5 years ago

I've noticed this new io_uring buzz going on, as it was released in Linux 5.1. They claim "up to two times performance gain on CEPH". It should improve multiqueue scalability for NVMe drives. Also the new API is supposed to be easier to use than currently used AIO API. So naturaly as enthusiastic ZFS user, i can't do better but ask what are the consequences for ZFS on Linux.

Is it likely that we will get some performance boost from this? Is this hard to implement at ZoL side?

behlendorf commented 5 years ago

This is definitely an interface we'll want to investigate supporting.

Harvie commented 5 years ago

Update: https://www.phoronix.com/scan.php?page=news_item&px=IO_uring-755x-Performance-Fix https://www.phoronix.com/scan.php?page=news_item&px=Linux-io_uring-Fast-Efficient

JonKohler commented 4 years ago

Stumbled upon this today, has anyone given this a poke?

A few interesting thoughts - now that 5.4 long term is out and cooking, more people may feel comfortable getting on to a long term version of linux that has io_uring. Secondarily, Ubuntu 20.04 said that they're standardizing around 5.4.

The "buzz" @Harvie mentioned seems to still be buzzing

Harvie commented 4 years ago

I think i still do not fully understand how io_uring works... Initialy i was thinking that io_uring would lay between ZFS and harddrive, but now somebody told me it is gonna lay between ZFS and userspace application, so i am bit confused. Do you know more about how this actualy works?

maxximino commented 4 years ago

Is this actually relevant for ZFS? AFAIU io_uring is an interface between userspace applications and the kernel. ZFS data paths between userspace and the kernel that I'm aware of are:

Filesystem IO - that's an interface managed by the kernel, and I can't spot in kernel 5.7 fs/ dir any filesystem-specific io_uring code. Looking deeper, https://github.com/torvalds/linux/blob/1ee08de1e234d95b5b4f866878b72fceb5372904/fs/io_uring.c#L2607 should call https://github.com/openzfs/zfs/blob/657ce253575295ef680eb33cf4ef698548212a46/module/os/linux/zfs/zpl_file.c#L289-L302 (and it has a fallback in case we didn't implement it - so looks like doesn't expect integration work for every FS)
Block devices/zvol IO - same as above, checked drivers/block/, no sign of anything special . Didn't follow the exact code path for block devices.
ZFS send/receive: AFAIU that's one long-running IOCTL which dumps data directly into a file descriptor, so might not benefit from the reduced context switching of io_uring. (but this is just my speculation)

Ceph instead got huge performance gains because Ceph's ""backend"" is an userspace application which issues a lot of I/Os (see https://github.com/ceph/ceph/pull/27392). Hypotethically, if ZoL was not a native kernel module but still zfs-fuse, that would gain performance.

sistemas-sn commented 2 years ago

Hi, how are you? any news about this?. This will be a great feature for improves zfs performance.

recklessnl commented 2 years ago

It's coming up to 3 years since this post - @behlendorf has there been any progress or updates with regards to io_uring?

gigatexal commented 1 year ago

BRTFS -- a COW fs -- has seen improvements from io_uring :shrug: https://www.phoronix.com/news/Linux-6.1-Btrfs

KungFuJesus commented 1 year ago

BRTFS -- a COW fs -- has seen improvements from io_uring 🤷 https://www.phoronix.com/news/Linux-6.1-Btrfs

This was an improvement to how async io was handled when coming from io_uring rather than utilizing io_uring itself.

ghost commented 1 year ago

OpenZFS supports io_uring, verified by the test suite: https://github.com/openzfs/zfs/blob/master/tests/zfs-tests/tests/functional/io/io_uring.ksh

rincebrain commented 1 year ago

I don't...think this should be closed, unless I'm grossly misunderstanding OP's goal.

That check just confirms that fio using its io_uring engine doesn't error out. That doesn't say anything about ZFS actually leveraging this interface in any way - which it does not, as of this writing.

Feel free to close this again if I'm mistaken, but "there's a ZTS test that fio using io_uring can use OpenZFS" and "OpenZFS leverages io_uring in any way, or decides it's not useful" don't seem like the same goal.

wibed commented 1 year ago

i could imagine zvol iops and io_uring are a good match has anyone tested this, or any experience around it?

https://github.com/openzfs/zfs/issues/11407

Harvie commented 1 year ago

https://www.phoronix.com/news/Linux-6.6-IOmap-Improvements

Does ZFS on Linux use iomap? There is some iomap improvement that leverages io_uring and memory folios to supposedly improve performance of filesystems which are using it.

https://lwn.net/Articles/935934/

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6016fc9162245c5b109305841f76cca59c20a273 https://lore.kernel.org/io-uring/20230720181310.71589-1-axboe@kernel.dk/

openzfs / zfs

Support for io_uring API? #8716