FDT is an Application for Efficient Data Transfers which is capable of reading and writing at disk speed over wide area networks (with standard TCP). It is written in Java, runs an all major platforms and it is easy to use. FDT is based on an asynchronous, flexible multithreaded system and is using the capabilities of the Java NIO libraries.
By using O_DIRECT mode the kernel page cache is bypassed and transfers
can be DMA'd directly into user-space memory, and then directly DMA'd
out (on HW accelerated cards).
This means the CPU will never have to touch or copy the data files
being transferred, and will only be responsible for managing the channel
operations. Obviously, adding in a SW hash or checksum on transfers may
still cause CPU usage, but there will be one less copy of memory buffers
and the page cache itself won't get thrashed on large transfers (which
don't benefit from caching).
-edit- Files of any size can now successfully be written in O_DIRECT mode, so patch is complete.
By using O_DIRECT mode the kernel page cache is bypassed and transfers can be DMA'd directly into user-space memory, and then directly DMA'd out (on HW accelerated cards).
This means the CPU will never have to touch or copy the data files being transferred, and will only be responsible for managing the channel operations. Obviously, adding in a SW hash or checksum on transfers may still cause CPU usage, but there will be one less copy of memory buffers and the page cache itself won't get thrashed on large transfers (which don't benefit from caching).
-edit- Files of any size can now successfully be written in O_DIRECT mode, so patch is complete.