Windows version?

[lonnietc]

Hello,

I am seeking a cross-platform (Windows, Linux, OSX) version of a virtual switch with high-throughput, low-latency, and massive-scaling potential and am wondering what it might take to make rvs work towards this goal.

Any information would be greatly appreciated. Thanks

[yasukata]

Thank you for your message.

Regarding the Windows version, I think that compilation itself is not too complicated; I could have compiled the code in this repository using Visual Studio 2022 on Windows while I needed minor changes because of C++-specific pointer casting, the attribute syntax, and inline assembly code.

On the other hand, I am not sure if this is sufficient for your requirements, and it would depend on the environment that you think of; for instance, a virtual switch may be for steering packets for VMs running on Hyper-V or VMware, or we could make a software-based switch box (machine) by executing a virtual switch implementation on a physical machine having many physical NIC ports.

In particular, if you plan to use a virtual switch for VM networking, consideration for the virtual NIC representation would be necessary because OSes on VMs must have specific device drivers for virtual NICs. Regarding this VM-side device driver issue, I think virtio-net is one of the most common virtual NIC representations, and many existing OSes already have the virtio-net device drivers; therefore, if the VM networking system is compatible with virtio-net, OSes on VMs could easily use the provided virtual NICs. On the other hand, the implementation in this repository defines its own virtual NIC representation and is not compatible with virtio-net. Consequently, OSes on VMs cannot use our special virtual NICs if we do not provide the device drivers of our special virtual NICs for the OSes, and currently, I only have a device driver for DPDK ( https://github.com/yasukata/librte_pmd_rvif ). I thought this part may conflict with your requirements.

Essentially, I think an appropriate virtual switch implementation would depend on the targeting use case; if it is for VM networking, the virtual NIC representation of this repository may not satisfy your criteria.

Thank you very much for your interest. Please feel free to add information/comments to this issue if you wish to.

[lonnietc]

Hello @yasukata and Thanks for you detailed information posted here.

So as to give you a little background on the project, I am in the planning stages for a cross-platform (Windows, Linux, MacOS) highly-performant "P2P Relay Mesh" dynamic network that can scale to extremely large sizes.

For all of that long description, what I started thinking about was to look into the possibility of finding, or developing, in C/C++ or Rust, a type of OSI Level-2 vSwitch that could be run on various nodes and allow users to connect as needed.

Ultimately it might be akin to something like Yggdrasil (https://yggdrasil-network.github.io) but as a L2 switch that could perhaps integrate Boringtun or Wiregard for the encryption and also possibly use WinpkFilter (https://www.ntkernel.com/windows-packet-filter/) for which there are C/C++ and Rust implementations, or Windivert for the networking interface on Windows and maybe something similar for Linux/MacOS, but am still looking into options.

The core requirements are to:

1. Have "near-native" throughput speeds and low-latency
2. Ultra-high scalability
3. Encryption
4. Cross-Platform capabilities (maybe not at first)

Effectively, the idea came that instead of just writing another VPN, as there are so many out there, that going deeper down the network stack to perhaps develop some type of virtual-switch that would pass just about any traffic since ultimately this was about trying to see if it is possible to create a completely viable sub-Internet as a new P2P solution.

Another thought was to figure out a way to have everything working and just possibly take advantage of the IPv6 capabilities as well since the "new" protocol is supposed to handle Uncast, Multicast, and Allcast very well but that is really more of a Layer-3 type thing.

I researched VALE/Netmap, Snabb, and a number of others just to get an idea what may be possible and found that Snabb uses a LuaJIST which is pretty interesting although does not run on Windows, and the VALE/Netmap has Linux and Windows capabilities but they are VERY dated and would really require heavy development to make them work.

Because of this, I started seeking some other options since it could require a long time to rework those efforts and maybe was not the best approach to proceed. Then I cam across rvs and was thinking about a vSwitch possibility again.

These are just some thoughts.

[yasukata]

Thank you very much for providing the information.

I am sorry that I am not familiar with P2P networks, but please let me write some comments.

The first thing I thought is that to achieve the best performance, the use of one of the fast packet I/O frameworks such as DPDK and netmap is essential because they usually have a huge performance advantage over the traditional I/O subsystems provided by the widely used OSes.

On the other hand, I also thought that if you employ the existing Wireguard subsystems (e.g., the device driver implementation merged in the Linux kernel), the components involved for Wireguard, particularly 1) tunnel network devices exposed to users and 2) TCP/IP stacks needed for sending/receiving UDP packets, would become the bottleneck and the virtual switch may not be a significant bottleneck. This means that even if we could have an extremely fast virtual switch with a fast packet I/O framework, the performance can be limited by the components necessary for Wireguard.

The reason for the thought above is that according to the paper about netmap ( https://www.usenix.org/conference/atc12/technical-sessions/presentation/rizzo ), it looks difficult for TCP/IP stacks implemented in widely used OSes to achieve high UDP throughput close to 10 Gbps with small packet sizes; to overcome this limitation, netmap bypasses 1) the traditional network device I/O interface and 2) the TCP/IP stack of the OS. On the other hand, for example, the Wireguard device driver in the Linux kernel is tightly coupled with 1) the network device framework defined by the Linux kernel for creating the interface exposed to users and 2) the TCP/IP stack of Linux for sending/receiving UDP packets, which are the components that netmap decides to bypass for performance. Therefore, I expect that even if a virtual switch, leveraging a fast packet I/O framework, is employed, the entire performance can be limited by the components necessary to execute Wireguard which are the traditional subsystems of OSes.

Besides the performance issue, for the compatibility with the existing networking subsystems of OSes, I think it is better to follow the network device frameworks provided by OSes (e.g., the tunnel network device of Linux used for Wireguard).

Because of these reasons, I think it may be OK to start without a fast packet I/O framework, and in that case, using or extending the Open vSwitch implementation would be an option because Open vSwitch seems to have support for Windows.

As I noted, I am not familiar with this topic, but I wish these comments could be of any help. Thank you very much for sharing your idea.

[lonnietc]

Hello My Friend,

The use of Wireguard is only one option since I am still researching the whole idea more and it could potentially turn out that it is not the best solution to go with for the project.

You definitely raise some very good points to be sure and I did have a bit of curiosity about the bottleneck being somewhere besides the actual vSwitch which could be a problem.

To that end, I have also been looking into kernel-bypass for I/O and networking, in particular. I ended up coming across "Demikernel" (https://github.com/microsoft/demikernel) and (https://www.micahlerner.com/2021/11/09/the-demikernel-datapath-os-architecture-for-microsecond-scale-datacenter-systems.html) which is a Microsoft repository that is supposed to build for Linux and Windows, I think. (I attached a paper here for you also)

In general, what I get out of the Demikernel, and similar, kernel-bypass approach is that the software would effectively take control of the Ethernet card and not pass anything through the kernel which slows things down significantly.

Perhaps, with something like this, then a really nice system could be established. Maybe !!!

demikernel-sosp21.pdf

See: https://medium.com/@penberg/on-kernel-bypass-networking-and-programmable-packet-processing-799609b06898

Which also discuss kernel-bypassing (i.e. Demikernel layer) as an interface to the RDMA and DPDK interfacing to the nic.

Just some really crazy ideas here, but I think but they open a wide range of possibilities for some amazing cross-platform vSwitch development.

Also wanted to say Thanks for the thoughts and discussion on this.

[yasukata]

Thank you for your message.

Demikernel looks quite nice especially because its goal is portability; it would fit your targeting use case.

Anyway, your vision sounds very cool. I hope you will reach/create a set of technologies to realize your idea.

I learned new things from this discussion, particularly, from your idea. Thank you very much.

[lonnietc]

Hello and I hope that your weekend is going well.

I too have very much enjoyed this discussion since it opens up some intriguing possibilities.

I am wondering if you you might have an interest in working on a project towards something like this idea using your rsv to see if an ultra performant vSwitch could be constructed that also just might use something like demikernel or similar for a truly cross platform approach.

I have also previously investigated "Open vSwitch" and it is not as cross-platform as one would hope which seems more problematic to get working on Windows (and others as well) than to just start fresh either from "scratch" like building from rsv, or perhaps taking the VALE/netmap or SNABB with the LuaJIT approach, and trying to build a cross-platform version. I generally like to try an keep things simple whenever possible, but think that rsv and demikernel could potentially be be interesting. One catch here is that I think that Demikernel is in Rust and your rsv is in "C' but that is just a detail at this stage.

It would be very cool to develop a really awesome vSwitch that would meet many users needs as well as my needs on the P2P project.

Have a great day and please let me know if you are interested in working on something like this for which we can connect on a small Slack channel that I have to discuss things more to see what could be done, okay.

[yasukata]

Thank you very much for your kind suggestion.

Although your idea sounds cool, I am already working on several projects in parallel, and if I add more projects, the entire load will exceed my capacity.

Therefore, I am sorry, but I cannot join your project this time.

I appreciate your suggestion, and I wish your project to be fruitful.

[lonnietc]

Thanks for getting back to me and it is no worries as I completely understand.

I will keep you updated on the progress once the final plan is laid out for a direction to proceed but think that just maybe a vSwitch approach is the best way for this purpose.

Have a great day, my friend