search

microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.26k stars 812 forks source link

Enable VRF Support in WSL2 linux kernel #6850

Open abhishek-pandey-1 opened 3 years ago

abhishek-pandey-1 commented 3 years ago

Is your feature request related to a problem? Please describe. My team in Azure is trying to adopt WSL2 as our dev environment. Our dev work and tests depend on linux VRFs but this isn't currently included in the WSL2 linux kernel build.

Describe the solution you'd like I'd like a straight forward way to enable VRF support the WSL2 linux kernel. This can be:

  1. VRF support enabled by default
  2. Ability to compile and insert vrf module into the shipped kernel easily.

Describe alternatives you've considered I looked into compiling the VRF module and inserting it into the pre-installed kernel on boot. This has two problems:

  1. VRF depends on IPV6_MULTIPLE_TABLES which has to be a built-in module but isn't included in the default kernel.
  2. Even if the IPV6_MULTIPLE_TABLES was built-in, WSL2 at the moment doesn't support inserting kernel modules.

As a workaround, I have compiled a WSL2 linux kernel release with the config needed to support VRFs and replaced the default shipped kernel with this. This works Ok, but:

  1. It is not scalable -- We want to be able to allow developers to be up and running without going through this process.
  2. It is not maintainable -- We'll need to re-rollout this solution for each new kernel update.

Additional context

Enabling VRF requires this additional config:

CONFIG_IPV6_MULTIPLE_TABLES=y
CONFIG_NET_VRF=y
WSLUser commented 3 years ago

So 2 things:

1) Can you explain briefly what VRFs are and their purpose?

2) While module support doesn't exist yet for WSL2, you can still install your own modules. In fact modules make the running kernel much more performant as the modules only run when needed instead of the entire time a distro is up. https://wsl.dev/wsl2-kernel-zfs/ provides an excellent how to for compiling kernel and adding your own modules (in case of the blog, the zfs module is shown but it's the same steps still).

WSLUser commented 3 years ago

https://www.kernel.org/doc/html/latest/networking/vrf.html provides the info. I will add this to my PR.

abhishek-pandey-1 commented 3 years ago

Ok,

  1. VRFs (as described in the link you added) allow the kernel to have multiple routing tables. With VRFs, we can logically isolate interfaces and restrict traffic flowing between them. Our service/platform depends on VRFs to do exactly that.

  2. Yes, this is what I have done. The VRF module is a part of the linux kernel, so I was able to compile the kernel with this enabled and use it on my WSL2 instance.

The issue is that we have many people working on the core platform and we expect a lot more teams to start building services on top of our platform. Having all these people compile and replace the shipped kernel just to get started seems unnecessary.

The ideal solution to this would be to default the kernel config to set CONFIG_NET_VRF=m which will compile the VRF module but not insert it into the kernel by default. Then, a developer can simply do modprobe vrf to insert and activate the vrf kernel module. But since module inserts are not supported in WSL2, the only options are to either include CONFIG_NET_VRF=y in the default config or compile the kernel locally and use that instead of the kernel shipped with WSL2.

WSLUser commented 3 years ago

The issue is that we have many people working on the core platform and we expect a lot more teams to start building services on top of our platform. Having all these people compile and replace the shipped kernel just to get started seems unnecessary.

I would suggest publishing your custom kernel as a base as part of the service however WSL2 isn't intended to serve for production. I can understand using VRF for simulating a prod environment but WSL2 is intended for dev/testing. Your services should be deployed on a real Linux host which should already have VRF enabled as a module.

abhishek-pandey-1 commented 3 years ago

You are right. We are using WSL2 only for dev/test work. We need the VRFs to allow people to run tests on their machine.

The problem with publishing a custom kernel is that we will need to re-publish a custom kernel every time the WSL2 kernel is updated. This along with the fact that VRFs are a native feature of linux, it feels unnecessary to have to go through the trouble of publishing and maintaining a version of the WSL2 linux kernel that only adds VRF support.

At least in networking software industry, VRFs are widely used, so having it as a default option will likely be useful to others. Do you think having this enabled will have significant effect on the kernel performance? I would expect not.

stephencmorton commented 3 years ago

Agreed with everything @abhishek-pandey-1 says. I'm with a large group of developers who would find this very useful.

bradfitz commented 2 years ago

Tailscale also needs IPV6_MULTIPLE_TABLES. Currently we detect that we're running on a kernel like WSL with incomplete IPv6 support and just turn off IPv6 altogether, which isn't ideal.

We occasionally get support requests or tweets (e.g. https://mobile.twitter.com/false_temp/status/1464171823374151681) asking why IPv6 doesn't work in WSL.

/cc @apenwarr @danderson @crawshaw