API for sendfile

[petergsnm]

Can the libuinet provide an API for "sendfile". The FreeBSD used by libuinet has the support for sendfile, so it will be good to help sendfile implemented in libuinet.

http://www.freebsd.org/cgi/man.cgi?query=sendfile&sektion=2

[caladri]

I can't speak for Pat, but I wonder if you could say more about what the value of sendfile(2) would be for you in this case. That is, if you have a file in RAM and have a zero-copy sockets path into uinet, then it doesn't seem to have value. The primary value I'm familiar with for sendfile(2), at least, is that you can go directly from the buffer cache/VFS layer to the network without a copy, and sharing the backing object. The uinet model seems so different to me, that even supporting sendfile(2) seems like a mistake to me because it gives the wrong impression about where the tradeoffs are, but that may come from my working with e.g. network processors where the massive performance gains are very particular to how you work with data.

It seems like providing zero-copy write paths is probably more helpful, since you presumably have the file data you would want to send with sendfile(2) in userland already, otherwise it is not even beneficial, but harmful, to have to copy file data from the kernel just to pass it through a userland TCP/IP stack.

Or is it more a concern of application compatibility? In which case it seems like a suitably-pessimal wrapper implementation ought to be possible, but perhaps worth discouraging, again, because of the risk of giving the wrong impression of performance, semantics, etc. Note that, specifically, libuinet (per Pat's talk, as I recall/understand it) does not aim to be drop-in, but more like optimal and flexible, so that where an application really wants the in-kernel blocking API with all the things the kernel can do optimally, perhaps that application is not going to benefit from libuinet much.

[petergsnm]

Thank you.

As you rightly pointed out, this does not make sense when the data is available at the RAM. But, what if the data is available on a file system(if someone is doing a caching operation where the data is available on the file not in RAM)? In this case sendfile was helpful.

We can also solve the problem by mapping the file to the process memory and then use the libuinet to transfer the contents.

I also agree, libuinet might not be much to do for the implementation of sendfile. Can be done by the application.

[pkelsey]

I agree with @caladri that providing a zero-copy transmit path is the truly helpful to do here.

Given zero-copy transmit, I can still see the usefulness of providing a sendfile (or sendfile-ish) entry point as a convenience for adapting existing applications that are using sendfile to libuinet. The libuinet version would take care of mmap/munmap of the given fd and sending of the headers, data and trailers. I'm doubtful that the full range of sendfile behavior could be implemented (e.g., not sure how SF_NODISKIO could be respected), but certainly most of it and enough to accomplish what an application that wants to send file-system-resident data would be doing anyway, and certainly enough to avoid the pain of having to manually decompose sendfile calls in existing apps that are being adapted to libuinet.

[petergsnm]

Absolutely agreed.

But, improving the performance what we talked in the separate thread should be the first priority and then then sending side zero copy. This request can sit on back bench till then.

I have tested libuinet as well as the rump kernel and have got better results here and hoping it will improve in the days ahead.

Great work guys. Thanks to you all.