search

WukLab / LegoOS

Disseminated, Distributed OS for Hardware Resource Disaggregation. USENIX OSDI 2018 Best Paper.
http://LegoOS.io
GNU General Public License v2.0
482 stars 73 forks source link

Replacing IB with Eth #21

Open sarsanaee opened 3 years ago

sarsanaee commented 3 years ago

Hello @lastweek

I'm interested to replace IB with Eth.

What I'm having in mind is to just support running Lego OS components on virtual machines separately and each of them talks to others using an Eth driver like e1000e and then throughout the virt-io in the host. For example in a 1P-1M-1S config we work like pcomponent -> eth -> virt-io -> host -> virt-io -> eth -> mem-component or storage-component

Do you think this is something we can achieve with some minimal implementation effort?

Thanks, Alireza

lastweek commented 3 years ago

Hi @sarsanaee ,

Thank you for your interest in LegoOS. The short answer to your question is Yes, we are able to replace IB with Eth and run that on top of qemu e1000.

The current repo has an e1000 driver, tested on QEMU.

To achieve the replacement, i have the following items in mind (it might not be complete):

  1. Add e1000 driver init function to the boot sequence, so that it could be loaded
  2. Test the socket interface on top of the e1000 driver (this is optional)
  3. Build a set of RPC interfaces (esp send_and_receive) on top of e1000 driver interfaces (or socket interface).
  4. Disable the IB polling threads.

The heavy-lifting part would be item 3. You can either build the RPC APIs on top of e1000 driver interface, or add a middle socket interface.

Please let me know how it goes.

yiying-zhang commented 3 years ago

Hi @sarsanaee

After enabling e1000 driver, I think you could use socket directly via lwip (/net/lwip/api/sockets.c). I've tested this long time ago. So I can't guarantee that with the current master version it still works. But you can first try this instead of building your own network stack.

Yiying