search

microsoft / demikernel

Kernel-Bypass LibOS Architecture
https://aka.ms/demikernel
MIT License
902 stars 118 forks source link

Add a LibOS using XDP #957

Closed kyleholohan closed 2 months ago

kyleholohan commented 12 months ago

Context

DPDK has scoped support for specific NICs. If a NIC is unsupported through this avenue, current acceleration is limited to io_uring or raw sockets. Windows clients have no alternatives. Adding a new LibOS using Express Data Path could provide a fast alternative which does not require NIC support.

Proposed Solution

Add a new LibOS which uses XDP/BPF technologies to implement network I/O. OS-specific solutions are required: *nix platforms use BPF API and AF_XDP sockets (which may also benefit from io_uring). Windows platforms require XDP for Windows. Layering needs further consideration w.r.t. io_uring and BPF.

Alternative Solutions

RDMA/RoCE provide another avenue for acceleration, although the complexity is arguably much higher than XDP (not to mention NIC-specific support). Both solutions require OS-specific considerations: Windows supports a variant of libibverbs, but it's still divergent from the nix support. Registered I/O may provide an acceleration avenue on Windows with similar characteristics to io_uring or raw sockets on nix. "XDP for Windows" seems to be the spiritual successor to RIO. Viewing RIO as a companion to io_uring in providing acceleration alternatives to all OSes, XDP seems a more cogent/converged solution (even with different implementations for *nix/Windows).

ohault commented 10 months ago

This is by far my favorite feature.