How to use sendfile on tokio sockets

[tailhook]

Hi,

It turned out the approach I've used in tk-sendfile doesn't work in any useful way.

The basic issue here is:

• we must use as_raw_fd to get file descriptor from TcpStream
• we can call poll_write to find out whether socket is writable, but ...
• there is no way to call socket.io.needs_write() because io is private, and ..
• poll_write always returns Ready until need_write() is called

So the only way to use sendfile with current tokio_core is to reimplement TcpSocket yourself (right?)

There are couple of ways to fix it:

• expose need_write() for TcpStream, or PollEvented altogether
• implement TcpStream::sendfile directly on socket (but sendfile is different on different platforms)

Same applies to splice, sendmsg and maybe other ways to send data to raw socket (I'm not sure this makes sense for splice though).

Any thoughts?

[carllerche]

Could you elaborate on how sendfile would need to be implemented?

[tailhook]

Could you elaborate on how sendfile would need to be implemented?

I'm not sure I understand question. Using sendfile is similar to write or writev. Something along the lines of:

    fn sendfile<F: AsRawFd>(&mut self, file: F, offset: u64, size: usize) -> io::Result<usize> {
        if let Async::NotReady = <TcpStream>::poll_write(self) {
            return Err(::would_block())
        }
        let r = sendfile(self.io.as_raw_fd(), file.as_raw_fd(), offset, size);
        if is_wouldblock(&r) {
            self.io.need_write();
        }
        return r
    }

The stumbling block here is that sendfile is different in every other OS. But golang implements simplest possible system call on all systems, so we may choose this way too. (except, probably windows, I guess it should be cfg(unix))

[alexcrichton]

Oh oops I believe it was my intention to expose the need functions and I forgot to do so. We'll want to have lots of docs about what the function even is but I think it's fine to expose

[tailhook]

@alexcrichton should need_write be on AsyncWrite?

[carllerche]

Why not provide the necessary implementations for sendfile vs. expose need_write?

[carllerche]

There are only so many sys calls that can be made... extension traits could be used if needed.

[alexcrichton]

@tailhook no I don't think we'd need need_write on AsyncWrite, we dropped the poll_write method and that's very specific to TcpStream, not general write abstractions.

I don't personally think we should bind all the syscalls, as I think that's impossible in the limit of time.

[asomers]

The problem is that sendfile isn't really asynchronous. On Linux, even if the socket is non-blocking, sendfile may block on disk I/O. There's no way to prevent it from blocking on disk I/O.

On FreeBSD with UFS it's better, because sendfile will actually return early and do the disk I/O in the background. But it won't do that with ZFS or any other file system.

[tailhook]

@asomers, this is a different problem. Yes, sendfile must be called in a thread. Along with open, stat and other filesystem-related syscalls.