microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.53k stars 822 forks source link

WSL2 , problem with network connection when VPN used (PulseSecure) #5068

Open fibu79 opened 4 years ago

fibu79 commented 4 years ago

I'm using MS v. 2004 (build 19041) with UBUNTU linux on WSL2. When I don't use VPN on windows , everything is fine - I have internet connection on windows and wsl2 ubuntu. But when established connection via VPN (on windows) then on windows still is OK - I have both internet and vpn connection , but on Ubuntu there is no network connection at all (no internet , no vpn access). I suspect there is a problem with NAT (on Hyper-V default switch) Any idea what could be wrong ? Additionally: on wsl1 everything worked fine (also when VPN enabled)

Currently on wsl2 it looks like this : fibu@DESKTOP-3N4US3P:/mnt/c/Users/fibu2$ ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.30.123.209 netmask 255.255.240.0 broadcast 172.30.127.255 inet6 fe80::215:5dff:fe41:b550 prefixlen 64 scopeid 0x20 ether 00:15:5d:41:b5:50 txqueuelen 1000 (Ethernet) RX packets 263 bytes 27705 (27.7 KB) RX errors 0 dropped 1 overruns 0 frame 0 TX packets 223 bytes 34352 (34.3 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1000 (Local Loopback) RX packets 2 bytes 56 (56.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 2 bytes 56 (56.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

fibu@DESKTOP-3N4US3P:/mnt/c/Users/fibu2$ fibu@DESKTOP-3N4US3P:/mnt/c/Users/fibu2$ ping google.com ping: google.com: Temporary failure in name resolution fibu@DESKTOP-3N4US3P:/mnt/c/Users/fibu2$ fibu@DESKTOP-3N4US3P:/mnt/c/Users/fibu2$ netstat -rn Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 0.0.0.0 172.30.112.1 0.0.0.0 UG 0 0 0 eth0 172.30.112.0 0.0.0.0 255.255.240.0 U 0 0 0 eth0 fibu@DESKTOP-3N4US3P:/mnt/c/Users/fibu2$ cat /etc/resolv.conf nameserver 172.30.112.1 fibu@DESKTOP-3N4US3P:/mnt/c/Users/fibu2$

mjmdude commented 4 years ago

Any idea with wsl2 will be fixed to allow connectivity through vpns?

Chiramisu commented 4 years ago

This really needs to be fixed though please. Urgently! It's been an issue for a long time and I use WSL to maintain Linux servers over SSH as well as Git commands. Heck, you can't even apt update without this working correctly unless you disconnect from your VPN first.

Chiramisu commented 4 years ago

Possible duplicate of #416

eiva commented 4 years ago

Have same issue on Cisco AnyConnect..

dpalma9 commented 4 years ago

Same here. I had to install WSL 1 in order to be able to work with the VPN (Pulse Secure). None of the proposed solution worked to me. Please, let us know when the issue is fixed.

Edit: if you need any extra info of my system, just let me know.

robichaud commented 4 years ago

Having the same issue with Cisco AnyConnect

kkzz8888 commented 4 years ago

Having issue with Cisco AnyConnect as well. This really should be fixed, it has been months from what I can see.

sirskip commented 4 years ago

I'm also having the same issue with Cisco AnyConnect and it is frustrating as I was really looking forward to being able to directly use WSL2. I'm running the latest and greatest 20.04 Windows Build.

One thing to note is that I use Docker Desktop and have switched it over to use WSL2.
The odd thing is that Docker Linux containers running under WLS2 containers do not have the same issue as as the direct Ubuntu WSL2 install. My WSL 2 Docker Linux containers work without issue when on or off VPN. While out of scope for this issue, I also do not have any DNS or IP issues either with WSL2 Docker Linux instances that I do have with the Ubuntu WSL2 install.

For those looking for a workaround for the VPN issue, this is another option. And you do not have to do any special configuration changes to make it work. As long as you can install Docker Desktop, you can grab your favorite Linux Docker image, create a docker file, setup a volume/mount to a local Windows directory for ease of file transfer, start it, then exec into it. Once setup, its almost as easy as getting into a WSL2 Ubuntu instance.

vnijs commented 4 years ago

I noticed the same thing as @sirskip. I have tried most of the workarounds for this issue but none of them have worked. It does not seems to be just a DNS problem (https://github.com/microsoft/WSL/issues/4285) because I can't even connect to a remote server using the IP address. However, if I start a docker container and run that same exact command from a ZSH-terminal connecting to the server through SSH works just fine.

dpalma9 commented 4 years ago

I noticed the same thing as @sirskip. I have tried most of the workarounds for this issue but none of them have worked. It does not seems to be just a DNS problem (#4285) because I can't even connect to a remote server using the IP address. However, if I start a docker container and run that same exact command from a ZSH-terminal connecting to the server through SSH works just fine.

That's didn't work for me either.

carlosmorcerf commented 4 years ago

Having the same issue

craigloewen-msft commented 4 years ago

Hi folks, if you're seeing VPN issues can you please collect some logs for us to help us diagnose it? Full instructions can be found here on how to do so. Please make sure to reply here with a link to your feedback item!

marwatk commented 4 years ago

@craigloewen-msft Will the logs collect routing information from the VM and host? The issue (which I think is a duplicate of #4277) is that some corporate VPNs are configured to capture all routes and don't allow split tunneling. In this situation no Hyper-V network interface can talk over the VPN (or in my case to any host). Here's a good description, though in my case the workaround didn't work.

Docker gets around this with vpnkit and a socket to NAT the traffic through a userland process.

craigloewen-msft commented 4 years ago

@marwatk that's a great call out, we worked with the networking team to develop those logs so I'll double check if routing information is captured and pass along some of that insight.

dpalma9 commented 4 years ago

HI @craigloewen-msft Thanks for the answer. I will collect the logs and sent to you.

kevinruder commented 4 years ago

Am using anyconnect vpn and am also having the same issue

marwatk commented 4 years ago

@craigloewen-msft It appears my group policy doesn't allow me to turn on diagnostics and usage data. Let me know if there's anything manual I can provide that would help.

raafa001 commented 4 years ago

Good afternoon!

I'm using Big-Ip Edge Client. The situation is as follows: If I open my ubuntu distro before connecting to the VPN it works. After I connect, it works for a while, until it stops having access to the internet.

As soon as I disconnect the VPN, my ubuntu returns to access the network normally.

danielorozco06 commented 4 years ago

Hello, I am using Citrix Netscaler Gateway installed in my Windows Machine. While using VPN:

With WSL2 the file /etc/resolv.conf shows: image

With WSL1 the file /etc/resolv.conf shows: image

Commands in PowerShell: List Distributions and WSL version used: wsl --list –verbose image

Change WSL version on distribution: wsl --set-version DistroName NumberVersionWSL image image

pieterlukasse commented 4 years ago

WSL2 not working with Cisco AnyConnect (for any of the commands that need internet/network connectivity) :(

Converting back to WSL1 did "solve" the issue...

rofrol commented 4 years ago

Works for me with 2FA on Windows 10 with Ubuntu WSL2 using Cisco AnyConnect Secure Mobility Client (not from Microsoft Store).

PS C:\> netsh interface ip show interfaces
Idx     Met         MTU          State                Name
---  ----------  ----------  ------------  ---------------------------
  1          75  4294967295  connected     Loopback Pseudo-Interface 1
  8          35        1500  connected     Wi-Fi
 18          25        1500  disconnected  Local Area Connection* 1
 23          65        1500  disconnected  Bluetooth Network Connection
 10          25        1500  disconnected  Local Area Connection* 2
 21        6000        1390  connected     Ethernet 2
 12        5000        1500  connected     vEthernet (Default Switch)
 30        5000        1500  connected     vEthernet (WSL)

inside wsl2:

$ cat /etc/resolv.conf
# This file was automatically generated by WSL. To stop automatic generation of
this file, add the following entry to /etc/wsl.conf:
# [network]
# generateResolvConf = false
nameserver 165....
nameserver 165....
search ...
$ nslookup yahoo.com
Server:         192.168.0.1
Address:        192.168.0.1#53

Non-authoritative answer:
#...
Name:   yahoo.com
Address: 2001:4998:44:3507::8000
Name:   yahoo.com
Address: 2001:4998:24:120d::1:1

Notes

  1. Do not use wsl --shutdown. Use restart_wsl.ps1:
Restart-Service LxssManager
  1. To access internal domains in vpn you need to have only name servers from from vpn in /etc/resolve.conf and no name servers like 192.*.

  2. vEthernet (WSL) with metric 5000 Cisco AnyConnect to metric 6000 works. 1 and 6000 works too. 5000 and 5001 does not work. So no need for me to run

Get-NetIPInterface -InterfaceAlias "vEthernet (WSL)" | Set-NetIPInterface -InterfaceMetric 1

Only run cisco.ps1:

Get-NetAdapter | Where-Object {$_.InterfaceDescription -Match "Cisco AnyConnect"} | Set-NetIPInterface -InterfaceMetric 4000
  1. I use this code to start dns-sync.sh https://github.com/rofrol/dotfiles/blob/f6e8c4fe8d3f5762a2f406658e25d4e5e79dd04e/.bashrc#L434.

  2. Here is my modified version of dns-sync.sh which does not add name servers like 192.* https://github.com/rofrol/dotfiles/blob/f6e8c4fe8d3f5762a2f406658e25d4e5e79dd04e/bin/dns-sync.sh#L51

  3. I didn't have to set mtu. Here it also says that mtu isn't always needed https://www.reddit.com/r/bashonubuntuonwindows/comments/gvaazf/wsl2_not_working_with_vpn_connection/ftmj20u/?utm_source=reddit&utm_medium=web2x&context=3.

  4. You can automatically start cisco.ps1 to change metric like here https://github.com/microsoft/WSL/issues/4277#issuecomment-639460712. But it wasn't working on my company's machine because of privileges. Also 30 seconds maybe not enough if you need to also enter token for 2FA.

  5. If you can't start automatically cisco.ps1, you can make shortcut to C:\Users\user\Desktop\cisco.ps1. Then change target to powershell.exe -f C:\Users\user\Desktop\cisco.ps1, click Advanced and set Run As Administrator.

  6. Make the same shortcuts for restart_wsl.ps1.

  7. Didn't work for me with Cisco AnyConnect from Microsoft Store.

  8. Sometimes I need to restart wsl service to have internet working.

  9. Docker is using some clever tricks with nat https://github.com/microsoft/WSL/issues/5068#issuecomment-681138926.

arcsector commented 4 years ago

I verified my resolv.conf setup works just fine by switching back to WSL 1 while on Pulse Secure VPN and making sure everything works as expected:

PS> wsl --set-version kali-linux 1
PS> wsl
$ cat /etc/resolv.conf
nameserver 1.1.1.1
$ nslookup google.com
Non-authoritative answer:
Name:   google.com
Address: 142.250.68.14
Name:   google.com
Address: 2607:f8b0:4007:811::200e

This shouldn't be this expensive to change; why does this work in WSL 1 but not in WSL 2?

c56pony commented 4 years ago

Following @rofrol 's information, I changed the Cisco AnyConnect metric to 6000, but it didn't work. My environment is below.

Ubuntu on WSL2 (metric is 5000)
Cisco AnyConnect Secure Mobility Client ver. 4.8.03052

I have failed, but I will describe the steps I took for reference.

  1. Launch Cisco Anyconnect and change the metric to 6000 in powershell.
    > Get-NetAdapter | Where-Object {$_.InterfaceDescription -Match "Cisco AnyConnect"} | Set-NetIPInterface -InterfaceMetric 
    6000
  2. Launch Ubuntu on WSL2 and configure dns-sync.sh.
    # wget https://raw.githubusercontent.com/rofrol/dotfiles/f6e8c4fe8d3f5762a2f406658e25d4e5e79dd04e/bin/dns-sync.sh -O /etc/init.d/dns-sync.sh
    # chmod +x /etc/init.d/dns-sync.sh
    # unlink /etc/resolv.conf
    # service dns-sync.sh start
  3. Restart WSL2
    > Restart-Service LxssManager
NiklasBr commented 4 years ago

I couldn't get @rofrol's solution working either, it looked promising, but no success with AnyConnect Secure Mobility Client 4.8.03052

rofrol commented 4 years ago

It is very poor situation with networking in WSL2.

  1. Event when it works for me, does not work for everybody.
  2. Sometimes requests from web browser are blocked to server in WSL2. GET request pass, but POST request with OPTION request before are sometimes blocked for the first time. I need to refresh page and they pass.
  3. My server in WSL2 cannot access another server in host.

I suggest switch to

  1. WSL1
  2. docker bc they do something differently with NAT https://github.com/microsoft/WSL/issues/5068#issuecomment-681138926
ronald-willems commented 4 years ago

I solved this for my situation by establishing the VPN connection not from the windows host, but from within the WSL2 VM. Obviously I cannot use the vpn connection on the Windows side, but that's ok for me. I'm also using X forwarding with XMing to be able to use a browser over de vpn connection

dpalma9 commented 4 years ago

I solved this for my situation by establishing the VPN connection not from the windows host, but from within the WSL2 VM. Obviously I cannot use the vpn connection on the Windows side, but that's ok for me. I'm also using X forwarding with XMing to be able to use a browser over de vpn connection

So did you have to configure the proxy on the WSL2 VM? Or what did you do exactly?

ronald-willems commented 4 years ago

I will be a bit more specific.

First stop vpn on your windows machine. Go to WSL and type: sudo apt-get install openconnect sudo openconnect [SERVER] -b -u [USERNAME] --authgroup [AUTHGROUP]

That's it. No proxies or other things.

If you can get openconnect to work for your vpn situation, you will have a vpn connection from your wsl vm/distro to your office network. On the windows side you will not have any vpn this way. I can imagine that's not good enough for everyone, but it might be for some.

aggroot commented 4 years ago

Here is a short video recording for a solution I found for this issue: https://www.youtube.com/watch?v=yR2NsssY7z8 . More details about my findings here: https://github.com/microsoft/WSL/issues/4246#issuecomment-691561185 Basically, whenever you want to connect to vpn, before connecting to VPN, make sure you disable WSL2 network and reenable it afterwards

dpalma9 commented 4 years ago

I will be a bit more specific.

First stop vpn on your windows machine. Go to WSL and type: sudo apt-get install openconnect sudo openconnect [SERVER] -b -u [USERNAME] --authgroup [AUTHGROUP]

That's it. No proxies or other things.

If you can get openconnect to work for your vpn situation, you will have a vpn connection from your wsl vm/distro to your office network. On the windows side you will not have any vpn this way. I can imagine that's not good enough for everyone, but it might be for some.

I have to use another VPN client but maybe it works too. I'll try it and let you all know.

danydc commented 4 years ago

Here is a short video recording for a solution I found for this issue: https://www.youtube.com/watch?v=yR2NsssY7z8 . More details about my findings here: #4246 (comment) Basically, whenever you want to connect to vpn, before connecting to VPN, make sure you disable WSL2 network and reenable it afterwards

Works perfectly! Thanks!

klikevil commented 4 years ago

Also effected by this issue, when I launch openconnect from within the wsl instance and add a static route back to my local machine I can access resources (e.g. connecting to an xserver that i run on the host os). The inverse doesn't work (e.g. i can't connect cisco anyconnect launched on host from the guest). This is not a nameserver issue from resolv.conf you can't ping IPs at all; tried all solutions available none worked.

gerardbosch commented 4 years ago

Hi there, the proposed workaround of disable/enable WSL network adapter does not work in my case with Citrix Netscaler Gateway VPN :( Waiting for an official solution here.

ronald-willems commented 4 years ago

For me the disable/enable WSL network adapter doens't work either in combination with cisco anyconnect. Also my own solution (see above) of only using vpn from within wsl is not good enough. I found out that I do need vpn on my windows side. So I'm back to WSL 1 and waiting for an official solution as well

abarke commented 4 years ago

I am having a similar issue when I'm on the GlobalProtect VPN connection to our corporate network. One workaround I've found is to add the IP for your router to /etc/resolv.conf as a nameserver entry.

We are also experiencing issues with our corporate managed GlobalProtect VPN. However I cant even ping 8.8.8.8 when connected to VPN, nor can I access any external or internal IP directly. If using IP address directly doesn't work, then I assume that this isn't a DNS issue, rather an underlying networking issue?

mjmdude commented 4 years ago

I went back to WSL1. Everything works now. I will upgrade to WSL2 once it is fixed.

marwatk commented 4 years ago

We are also experiencing issues with our corporate managed GlobalProtect VPN. However I cant even ping 8.8.8.8 when connected to VPN, nor can I access any external or internal IP directly. If using IP address directly doesn't work, then I assume that this isn't a DNS issue, rather an underlying networking issue?

Depending on your VPN setup a bunch of people are having different issues. Some are simple resolv.conf issues, these people are likely using a VPN that supports split tunneling and some minor changes will fix them.

But the major issue (and the one I'm experiencing) is that some VPNs can be configured to capture all routes which breaks routing between the Hyper-V VM and the host. (We saw this at our corp on Cisco AnyConnect, but we've since migrated to GlobalProtect and experience the same issue).

If you try to start a Hyper-V vm using NAT while connected to the VPN you'll see similar behavior. The guest can't talk to anything (even the host) because when the host tries to respond the traffic is routed through the VPN instead of the NAT interface.

Docker gets around this by having a userland proxy (vpnkit) that uses a Hyper-V socket to tunnel traffic. Until WSL 2 (or a third party) implements something similar I don't see a viable solution.

Some people report that manually altering routes or tweaking the order of operations when starting the VPN will work, but in our case the VPN software is actively monitoring the routing table for changes and will revert them to prevent leaking traffic.

Edit: @sakai135 has hacked together vpnkit and WSL 2 as a workaround

lpuglia commented 4 years ago

@marwatk thanks for the comment, can you tell me why wsl1 doesn't have the same problem?

aelray commented 4 years ago

@marwatk thanks for the comment, can you tell me why wsl1 doesn't have the same problem?

Because wsl1 doesn't run inside a VM

dalgibbard commented 4 years ago

I'm using the documented workaround here, works great! https://github.com/microsoft/WSL/issues/4277#issuecomment-696246606

kellycouch commented 4 years ago

None of the proposed solutions I've tried have resolved the issue. Using AnyConnect VPN and WSL2 Windows build 18363.1082 Unable to ping any IP outside the WSL, on VPN or localhost.

neilpalima commented 4 years ago

fix this please

cgbur commented 4 years ago

I can't connect to my vpn server using the windows store version because it doesn't allow you to add any custom options besides the address... If anyone knows how to set a group let me know.

lachlansimpson commented 4 years ago

Other people have noticed Linux under WSL2 can be leaking

beejaz commented 4 years ago

WSL2 with Cisco AnyConnect (from Windows Store version) now works for me and alternative is to use OpenConnect as I replied here https://github.com/microsoft/WSL/issues/4277#issuecomment-703070493

SpencerDawson commented 4 years ago

@cgburgess take a look at my earlier comment.

https://github.com/microsoft/WSL/issues/5068#issuecomment-646386328

with Anyconnect, you only provide the host and a name for it. When you connect, it will prompt you, like so: image

If you need to configure it further, then you likely need a client other than Anyconnect.

philippe-granet commented 4 years ago

I confirm that this hack of vpnkit and WSL 2 from @sakai135 works for problem with network connection when VPN used (PulseSecure): https://github.com/sakai135/wsl-vpnkit

AlbesK commented 4 years ago

I have the same issue, using Cisco AnyConnect.

The quickest solution to not break the current setup I have is reverting back to WSL1 using PowerShell for the distro through wsl --set-version distro 1, where distro is the exact name you see for the Linux distro from the wsl -l -v command to list distros and versions including docker-desktop integration etc. You can find the command with wsl --help as well. It took about 3-5 minutes but it might vary for you.

This saves the hassle on downgrading all of wsl to the 1st version, and this means that you can still use version 2 on any other installed distro if you so wish or upgrade up again by inserting 2.

Lastly for now, the Microsoft AnyConnect version that is being mentioned in this thread and others is slightly different than the internal Cisco one.

androidotai commented 4 years ago

I have the same issue, using PulseSecure client from store.

My case is totally weird. Both of my host connection, either without VPN or with VPN can connect to internet. But the weird part is, the internet from my WSL2 only available when my host connect to VPN. when I disconnect host from VPN, no internet connection from my WSL2. Anyone facing same case like me?

dcopenhaver commented 4 years ago

Downgrading to wsl v1 - however I'm very close to the next step being to just REMOVE every single feature hyper-v related and go back to something that 'just works', VirtualBox. I like the WSL idea (a lot!), and I like that MS is doing it, but I can't spend any more time on it's problems, I have work to do. I gave VirtualBox up because I wanted WSL v2, which also means I was FORCED to also use hyper-v for my full blown VM's.... nothing but problems on all accounts really. From issues with 'enhanced session mode' for linux VM's (which is absolutely mandatory or the VMs are unusable) to these network issues with WSL. Very disappointing... and this particular issue has been open since APRIL!?