axboe / liburing

Library providing helpers for the Linux kernel io_uring support
MIT License
2.72k stars 393 forks source link

copy_file_range op? #831

Open SUPERCILEX opened 1 year ago

SUPERCILEX commented 1 year ago

There's splice but no copy_file_range. It'd be great to have.

axboe commented 1 year ago

copy_file_range() just uses splice underneath the covers for the generic implementation anyway, can't you just use that directly from io_uring as well?

If not, it could certainly be wired up. But the majority of the work would be enabling nonblocking issue of it, otherwise you'd end up in the same situation potentially as your unlink issue.

SUPERCILEX commented 1 year ago

Splice only works if one of the FDs is a pipe according the docs, so that won't work.

axboe commented 1 year ago

Right, so you'd splice to a pipe, splice from a pipe. Not ideal, but it'd work.

SUPERCILEX commented 1 year ago

Won't that introduce an unnecessary copy?

vlovich commented 9 months ago

Yeah, I think a real copy_file_range implementation is required. Many filesystems these days support reflinks which copy_file_range would take advantage of while splice would generate a copy.

       copy_file_range() gives filesystems an opportunity to implement
       "copy acceleration" techniques, such as the use of reflinks
       (i.e., two or more inodes that share pointers to the same copy-
       on-write disk blocks) or server-side-copy (in the case of NFS).

So if I'm copying on the same XFS/BTFS/EXT4 filesystem (& recently ZFS too I believe), copy_file_range would copy a few inodes max regardless of the file size (+not actually consume that space) whereas splice would duplicate the entire file.

copy_file_range is only a splice under the hood when copying across filesystems. Within a single filesystem there's an accelerated path available.