search

ReturnInfinity / BareMetal-OS-legacy

BareMetal is a 64-bit OS for x86-64 based computers. The OS is written entirely in Assembly while applications can be written in Assembly, C/C++, and Rust.
1.74k stars 302 forks source link

Zero-copy in the network stack #43

Open IanSeyler opened 11 years ago

IanSeyler commented 11 years ago

http://en.wikipedia.org/wiki/Zero-copy

hidnplayr commented 11 years ago

Go for it! :)

benaryorg commented 9 years ago

So, you want a syscall (?) to send a memory range directly onto the network?

IanSeyler commented 9 years ago

Part of this has been implemented. os_ethernet_tx forwards the memory location of the packet directly to the network driver and the hardware handles it. No time is spent by the CPU to copy anything.

os_ethernet_rx does one copy (from the OS packet buffer to the requested memory address) so that will need to be adjusted.

benaryorg commented 9 years ago

It is a buffer, it will be hard to impossible to remove that, unless you let the application specify its own.

IanSeyler commented 9 years ago

Yep, the thought was to add an option to specify the address - sort of like adding the network callback address.

benaryorg commented 9 years ago

Is the buffer used for more than one packet?

scherrey commented 9 years ago

Are we talking about on-nic memory? Does the driver support this? If so how much is available?

I did zero copy security systems in promiscuous mode back in my security days. The cards we used had significant onboard memory which we reviewed in place and decided which data to copy into machine shared memory and which to ignore.

The call back mechanism that exists should work well for such a design if the memory is as fast or faster than system RAM otherwise it's best just to copy to system shared memory. Just need to ensure that your process talking to the nic always has a cpu assigned to it.

Is the buffer used for more than one packet?

— Reply to this email directly or view it on GitHub https://github.com/ReturnInfinity/BareMetal-OS/issues/43#issuecomment-120841007 .

IanSeyler commented 9 years ago

That buffer, os_EthernetBuffer, only stores the most recent packet that was received by hardware. It used to be a ring buffer but that was removed once the callback code was enabled.

We're talking about system memory here and the current drivers do not support on-nic memory.

Heres what is currently happening when a packet comes it:

1) Hardware grabs the packet and stores it in a system buffer. 2) App calls os_ethernet_rx to copy the packet in the system buffer to its own storage location. 3) Repeat

Here is what we want for zero-copy:

1) Hardware grabs the packet and stores it in a buffer specified by the application. 2) Repeat.

benaryorg commented 9 years ago

I think it would be an overall good solution if the application could specify it's own handlers for all interrupts (without overwriting the system ones).

That would solve many problems. It would enable writing a task-switching mechanism and would enable some better mechanisms than polling.

We also had exception handling then.

IanSeyler commented 9 years ago

At the moment the only two system interrupts are caused by the RTC and the network. Each of those supports an application defined callback so polling isn't required.

benaryorg commented 9 years ago

Sorry, but I haven't had time yet to have a deeper look into BareMetalOS.

Isn't there a Zero-Copy network "stack" then?

IanSeyler commented 9 years ago

At the moment BareMetal does not have zero-copy in its network stack.

Also I took a look at the existing network drivers and they contain the dredded rep movsb instruction. This change will involve driver modification as well. This is probably left overs from the ring buffer system that was removed.