search

cloud-hypervisor / fuse-backend-rs

Rust crate for implementing FUSE backends
Apache License 2.0
130 stars 66 forks source link

Will zero-copy read/write (splicing) supported? #17

Open zhangjaycee opened 3 years ago

zhangjaycee commented 3 years ago

From my understanding, it seems that splicing is not supported yet, will this be supported? Thanks!

jiangliu commented 3 years ago

Splice() requires that one of the two fds is pipe at lease. The fusedev and virtiofs transport layer can't satisfy the requirement yet. Recently there are some enhancements to the fuse protocol, which may help to do better than splice:)

zhangjaycee commented 3 years ago

Splice() requires that one of the two fds is pipe at lease. The fusedev and virtiofs transport layer can't satisfy the requirement yet. Recently there are some enhancements to the fuse protocol, which may help to do better than splice:)

Sounds great! Would you provide some clues about what is to be introduced to the fuse protocol? Do you mean this one, passthrough read/write (https://www.spinics.net/lists/linux-fsdevel/msg180464.html)?

For me, I think it's cool if virtio-fs can support splicing, or fusedev can support something like DAX.

jiangliu commented 3 years ago

Splice() requires that one of the two fds is pipe at lease. The fusedev and virtiofs transport layer can't satisfy the requirement yet. Recently there are some enhancements to the fuse protocol, which may help to do better than splice:)

Sounds great! Would you provide some clues about what is to be introduced to the fuse protocol? Do you mean this one, passthrough read/write (https://www.spinics.net/lists/linux-fsdevel/msg180464.html)?

For me, I think it's cool if virtio-fs can support splicing, or fusedev can support something like DAX.

yes, exactly! And we are investigating io uring, which seems promise to optimize fuse-backend-rs performance:)

liubogithub commented 3 years ago

@jiangliu Perhaps the splice() @zhangjaycee mentioned is supposed to work between /dev/fuse and files in the passthrough directory?

bsbernd commented 2 years ago

@jiangliu Out of interest, did you look into io_uring? From my limited understand of it, it might not be compatible. What we need is an SQ, filled in by the kernel and consumed by userspace and then the other way around for CQ. But afaik, io_uring only supports userspace filling SQ, with the kernel consuming it. I will try to go through the design details and code during the next days.

I had ioctl threads with per core queues as in zufs and Miklos fuse2 branch on our todo list, but then figured out about rather high waitq wakeup latencies.https://www.spinics.net/lists/linux-fsdevel/msg214814.html I experimental patches to allow kernel spinning and to reduce unneeded waitq wake (will send the patches today or tomorrow), but latency is still not ideal. The SQ/CQ approach might work better, at least if a user space consumer thread is already running.

jiangliu commented 2 years ago

@jiangliu Out of interest, did you look into io_uring? From my limited understand of it, it might not be compatible. What we need is an SQ, filled in by the kernel and consumed by userspace and then the other way around for CQ. But afaik, io_uring only supports userspace filling SQ, with the kernel consuming it. I will try to go through the design details and code during the next days.

I had ioctl threads with per core queues as in zufs and Miklos fuse2 branch on our todo list, but then figured out about rather high waitq wakeup latencies.https://www.spinics.net/lists/linux-fsdevel/msg214814.html I experimental patches to allow kernel spinning and to reduce unneeded waitq wake (will send the patches today or tomorrow), but latency is still not ideal. The SQ/CQ approach might work better, at least if a user space consumer thread is already running.

We have tried to enable io_uring for fuse-backend-rs but encountered some issues related to io-uring async io framework. And it's time to resume the async io support now:) It would be great to give a second try for splice when enabling async io:)