microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.25k stars 812 forks source link

[WSL2] Checkpoint VPN breaks network connectivity #4246

Closed rlipscombe closed 4 months ago

rlipscombe commented 5 years ago

(I've searched the open issues, and none that I could find were exactly the same)

Windows 10.0.18922.1000

I just installed Windows Insiders, and updated my Ubuntu distro to WSL2. It can no longer access the Internet.

From the Ubuntu bash prompt: ping github.com doesn't work (100% packet loss); ping 8.8.8.8 is the same.

/etc/resolv.conf gives nameserver 192.168.115.225. ping 192.168.115.225 doesn't work.

My Ubuntu distro has IP 192.168.115.230; I can ping that from Ubuntu.

The Windows IP address is 192.168.115.225, and I can ping it from PowerShell. Pinging the Ubuntu distro's IP (192.168.115.230) also works, from PowerShell.

Inside Ubuntu, route -n reports:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.115.225 0.0.0.0         UG    0      0        0 eth0
192.168.115.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0

I'm using a Surface Go, Windows 10 Pro, connected to the Internet over Wifi.

I might have some left-over detritus from when I attempted to get a Hyper-V VM connecting via Wifi. That was prior to upgrading to Windows Insiders. I don't know how much of that Hyper-V networking infrastructure is shared, and I don't know how to debug that.

rlipscombe commented 5 years ago

I attempted to convert the distro back to WSL 1, but it failed with The network connection was aborted by the local system.

rlipscombe commented 5 years ago

Oh, it might be worth noting that I've got Checkpoint VPN software (not active), Wireshark (i.e. npcap) and NordVPN (also not active) installed. I don't know whether any of those will break anything.

rlipscombe commented 5 years ago

Uninstalling NordVPN does not fix the problem.

The Checkpoint VPN software seems to be responsible for screwing it up. Uninstalling it fixes the problem.

Unfortunately (sigh), I have to have this software installed, so it looks like I'm going to have to uninstall Windows Insiders.

Any chance you could work with Check Point to get this resolved?

rlipscombe commented 5 years ago

So, interestingly enough, uninstalling and reinstalling the Checkpoint VPN software appears to fix the problem.

rlipscombe commented 5 years ago

(title updated to true cause of problem)

BenHenning commented 4 years ago

FWIW I've experienced what sounds like a similar issue, and I don't use Checkpoint VPN. I notice that when this happens, seemingly all socket-level operations seem to fail in Windows. Even my Android emulator becomes inaccessible to Android Studio, and all Chrome tabs indicate no internet connectivity. Closing all Ubuntu windows resolved the issue for me today, and this consistently happens when I leave a local server running in Ubuntu overnight and come back to my workstation 24 hours later.

cmeiklejohn commented 4 years ago

I'm using the Cisco AnyConnect VPN and as soon as I connect, I lose all access to the external network. Anything I can do to help debug this further?

craigloewen-msft commented 4 years ago

@cmeiklejohn please see issue https://github.com/microsoft/WSL/issues/4277

If you'd like to help us debug it please send us networking logs, instructions on how to do that are here!

neileadobe commented 4 years ago

I also have this problem, using Cisco. Logs here: https://aka.ms/AA6fthe

rlipscombe commented 4 years ago

Data point: with Windows 10.0.19013.1, CheckPoint VPN E81.40. If I right-click on the notification icon and select "Disable Security Policy" (thus regaining control of my own firewall) then WSL Ubuntu can connect to the Internet correctly.

jagjordi commented 4 years ago

Same issus occurs with Cisco OpenConnect VPN. Here are the logs https://aka.ms/AA6jmg1

timesnewmen commented 4 years ago

Similar issue with Citrix VPN. I can ping the server, but can not open tcp port 80 and curl is timeout.

codeart1st commented 4 years ago

Same issues also with Checkpoint VPN

caal-15 commented 4 years ago

Same problem with Cisco AnyConnect

elmorekevin commented 4 years ago

I lose internet connectivity in WSL2 when using SonicWall VPN in full-tunnel mode. If I switch to partial-tunnel, then WSL2 internet connectivity is fine.

wissamz commented 4 years ago

I am seeing the same behavior using Cisco AnyConnect VPN. Any updates on this issue?

iamoverit commented 4 years ago

same issue using Cisco AnyConnect (connected)

sphair commented 4 years ago

So, interestingly enough, uninstalling and reinstalling the Checkpoint VPN software appears to fix the problem.

I have the same problem, but this did not seem to help in my case.

hardik-id commented 4 years ago

I installed/used Cisco AnyConnect from Windows Store https://www.microsoft.com/store/productId/9WZDNCRDJ8LH and it started working. Credit goes to https://github.com/microsoft/WSL/issues/4277#issuecomment-561649724

andyneff commented 4 years ago

I have the same problem as @elmorekevin I'm using the latest Sonicwall NetExtender (9.0.274), and can only use full tunnel mode. WSL1 works perfectly at the same time WSL2 does not.

metawave commented 4 years ago

I have a similar problem with Citrix Netscaler VPN at work, which only tunnels some networks. Internet access is fine with wsl2 but connecting to a host inside a VPN tunneled network, the name can be resolved to an IP but then timeouts (wireshark says tcp retransmission). Citrix Netscaler says, that it has tunneled that connection in the "tunneled application" window. Also disabled the firewall completely, but that didn't work either....

andyneff commented 4 years ago

At random, I tried to use WSL 2 when I was connected to VPN, and to my utter and total surprise, it started working! I have not been able to reproduce the result since. But I was able to access both my VPN network and the internet (via full tunnel mode).

I did make an observation though. When it worked, I had done nslookup and run server and noted the IP address of the dns proxy server was 172.x.x.x. However other times (when it doesn't work) it's 192.168.x.x. (Now my real IP both locally and via VPN is 10.x.x.x subnets)

Sometimes I see three IPs in WSL2 (ifconfig), sometimes only two. I have no idea what is going on here. For example, now I only see 172.25.x.x and 127.0.0.1 (local host is always there), and it's not working. In my current example, I am able to ping the 172.25.x.x IP on my host windows machine, that is in the same subnet, but none of my other IPs

Recently updated to Windows 10 Pro build 10.0.19041

andyneff commented 4 years ago

Attempted to delete the WSL NIC/switch from hyper v fails (in a extremely bad way) I was hoping I could "reset the NIC" once connected to VPN by deleting it, and then letting it regenerate like it did the first time you run WSL2. It half deletes, and won't finish, and will never repair itself. I had to uninstall and reinstall WSL itself (not the distros)

andyneff commented 4 years ago

Workaround steps to get Internet working on VPN

Since the one time I got internet working on WSL2 was after an Windows 10 update, I was guessing that maybe somehow the network was reset, it and was because I started WSL2 while on VPN...

This has worked twice now using Sonicwall VPN, so I hope this works for someone else:

WARNING: You should always backup registry keys before you delete them, in case this breaks things!

  1. Remove the WSL Switch and NIC. Since neither WSL2 VM nor networks devices appear normally in Hyper-V Manager (which only hurts the users, so thanks), I cannot figured out how to use Hyper-V Manager to remove the Switch. It just errors out, and leave it broken. Now I found a Registry way to remove them
    1. Look in HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Class\{4d36e972-e325-11ce-bfc1-08002be10318}\####\NetSetupProperties, where #### is a four digit number
      • Out of the four digit keys in there, two of them will mention WSL
        • "NETSETUPPKEY_Interface_IfAliasBase"="vSwitch (WSL)"
        • "NETSETUPPKEY_Interface_IfAliasBase"="vEthernet (WSL)"
      • The two number should be consecutive. Delete both keys.
    2. Look in HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\vmsmp\parameters\NicList
      • Delete the Key containing "FriendlyName"="WSL"
    3. Look in HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\vmsmp\parameters\SwitchList
      • Delete the Key containing "FriendlyName"="WSL"
  2. Now reboot.
  3. Once you reboot, running ipconfig should no longer show Ethernet adapter vEthernet (WSL):
  4. VPN in (do not start WSL2 even once before doing this)
  5. Once completly connected to VPN, now start WSL 2
    • Enjoy internet (Until you have to do this all over again...)

While still on VPN, shutting down WSL2 and restarting it, still worked. However...

  1. wsl --shutdown
  2. Disconnect from VPN
  3. Reconnect from VPN
  4. Run WSL2 again

Does not work.


This is not a great workaround, but it is a start... Shortcuts welcome!

AmmarRahman commented 4 years ago

The workaround I have at the moment is to work within a container. Even though Docker uses WSL2 as it's backend, they seem to have got a better network setup that would work through the VPN.

metawave commented 4 years ago

I can confirm the comment of @AmmarRahman. After installing Docker Desktop on my Windows machine and switch to the WSL2 backend, I noticed that this docker daemon is able to access resources in the vpn (downloads an image from a docker registry there). I can also confirm it by running a container accessing resources on the vpn docker run alpine sh -c 'wget -O- https://some-vpn-internal.resource.com'. Eventhough the communication to vpn resources don't work in wsl2, ex. by running the docker wsl2 "machine" (wsl.exe -d docker-desktop). So I think something is actively preventing this to work

jparszuto commented 4 years ago

Same here - if I want networking in WSL2, I have to wsl --shutdown and disconnect from VPN. If I need to stay on VPN and want to access internal resources, I can only do it through docker container which is essentially running on Windows. Connectivity to internal resources in docker on the other hand, requires changing docker's default networking. Can't have it all, unfortunately!

AmmarRahman commented 4 years ago

That's strange. From within docker I have full access to internal resources without changing anything in docker network.

tremblaysimon commented 4 years ago

@AmmarRahman, I can observe the same behavior here. I'm using a VPN and a proxy and it works perfect only in a docker container (using Docker Desktop with WSL2).

I tried @andyneff workaround but didn't work unfortunately...

For now, my workaround will be using a container in WSL2 to be able to connect to network.

AmmarRahman commented 4 years ago

I have stumbled upon a solution on another issue. https://github.com/microsoft/WSL/issues/4698#issuecomment-628682785. sudo ifconfig eth0 mtu 1350 did the trick for me.

spaceraccoon commented 4 years ago

@AmmarRahman sadly it's not working for me - did you do any further configuration on the host side?

AmmarRahman commented 4 years ago

I haven't changed anything on the host configuration. However, I did run netsh.exe interface ipv4 show interfaces to verify that 1350 is the mtu that my VPN is operating at.

AmmarRahman commented 4 years ago

running netsh.exe from the host yields

Idx     Met         MTU          State                Name
---  ----------  ----------  ------------  ---------------------------
....
 20        5000        1500  connected     vEthernet (Default Switch)
 49        5000        1500  connected     vEthernet (WSL)
 23           0        1350  connected     Ethernet 3

However, ifconfig output is

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1350
        inet 172.26.53.72  netmask 255.255.240.0  broadcast 172.26.63.255

I hvae changed the mtu after I got connected to the VPN. I don't know if that makes a difference.

jparszuto commented 4 years ago

My mtu for VPN interfaces is 1500 which matches the mtu in WSL2 distro. Attempting to set mtu to 1350 froze my distro. I tried shutting down wsl (wsl --shutdown) and now when starting the distro it just comes with an error message The virtual machine or container was forcefully exited. [process exited with code 4294967295]. I was able to recover from it only by restarting PC.

AmmarRahman commented 4 years ago

Update: for some reason changing the mtu stopped working for me. Unfortunately, the windows store version (Capsule) does not work with our corporate setup so I couldn't test @spaceraccoon setup.

hertz1 commented 4 years ago

I have stumbled upon a solution on another issue. #4698 (comment). sudo ifconfig eth0 mtu 1350 did the trick for me.

Thanks @AmmarRahman, this worked for me!

rasschaert commented 4 years ago

The exact same issue occurs when using Forticlient VPN, the fix by @AmmarRahman also works in that case, with the same MTU value of 1350. Thank you!

I thought mention it because I found this issue through googling my symptoms and it might guide other people using the FortiNet / FortiGate / FortiClient VPN here.

cjchang1688 commented 4 years ago

VPN works with WSL 2 after uninstalled standalone Cisco Anyconnect and reinstalled Anyconnect via Microsoft Store.

jakawell commented 4 years ago

When I turn on my VPN (Cisco Anyconnect, non-Windows Store version) not only do I lose internet connection from WSL2, but my VS Code remote connection fails as well. It cannot reconnect again until I turn off the VPN. Similarly to others, I cannot use the Windows Store version due to our corporate requirements, so I hope whatever is making that work gets implemented into a fix on WSL2 soon.

abhijeetchopra commented 4 years ago

Replacing the SSL VPN client from Cisco AnyConnect to OpenConnect worked for a colleague.

testworksau commented 4 years ago

I can also only access the internet from within a docker container, within a WSL2 distribution, when connected to our Citrix NetScaler VPN.

@craigloewen-msft please let me know if I can help collect logs etc.

image

Once converted back to WSL1:

image

The routing table shown in WSL2 only has 2 entries for eth0; in WSL1 it has 66 entries which cover adapters named eth0,eth1,eth3,lo,wifi2.

timesnewmen commented 4 years ago

I can also only access the internet from within a docker container, within a WSL2 distribution, when connected to our Citrix NetScaler VPN.

@craigloewen-msft please let me know if I can help collect logs etc.

image

Once converted back to WSL1:

image

The routing table shown in WSL2 only has 2 entries for eth0; in WSL1 it has 66 entries which cover adapters named eth0,eth1,eth3,lo,wifi2.

Same issue here. I believe that's because Citrix NetScaler VPN has some very strange routing strategy.

cuichenli commented 4 years ago

Hey Glenn @testworksau , just enable the Local LAN access can fix this issue. image

But still, you can not access the internal network

daveomcd commented 4 years ago

When I turn on my VPN (Cisco Anyconnect, non-Windows Store version) not only do I lose internet connection from WSL2, but my VS Code remote connection fails as well. It cannot reconnect again until I turn off the VPN. Similarly to others, I cannot use the Windows Store version due to our corporate requirements, so I hope whatever is making that work gets implemented into a fix on WSL2 soon.

So I just worked with my IT Department on coming up with a fix. Here's what we found out and how we corrected it.

When you connect to Cisco Anyconnect WSL2 will NAT to the highest priority Ethernet Adapter. A newer version of Cisco Anyconnect from the Windows Store seems to correct this; however, it doesn't support Dual Factor Authentication. So to correct this we used a script that I saw mentioned elsewhere.

Steps:

  1. Restart Computer (or begin from fresh start)
  2. Login to Computer, Connect via Cisco AnyConnect
  3. Run a script from Powershell that includes the follow (Note: your adapter name may need to be changed on Line 2 in quotes)
Get-NetIPInterface -InterfaceAlias "vEthernet (WSL)" | Set-NetIPInterface -InterfaceMetric 1
$VPNInterfaceName = "Ethernet 3"
Get-NetIPInterface -InterfaceAlias $VPNInterfaceName | Set-NetIPInterface -InterfaceMetric 5001
  1. Once that's been completed open WSL and you should be good to go, unless there is something else causing further issues that I had not experienced.

I hope this helps, as we spent several days attempting to track down the cause.

Edit: Be sure if you have "Ubunut/WSL" set as your default profile in Windows Terminal, that you run the script prior to starting WSL. So if Ubuntu/WSL is your default, set your default to PowerShell, or switch over to PowerShell, run the command wsl --shutdown, then execute the above script, then reopen Ubuntu/WSL.

olsonnn commented 4 years ago

i have more or less the same issue (but vEthernet (WSL) is not in my list of adaptors)

Cisco anyconnect 4.9 + umbrella when vpn or management vpn starts it stops working (dns only, ping works) When i disable sentinelone firewall completely it also works. When i enable sentinelone firewall BUT allow all from any in<>out it does not work anymore.

Umbrella and Sentinelone are no good friends.

xomanova commented 4 years ago

The workaround I have at the moment is to work within a container. Even though Docker uses WSL2 as it's backend, they seem to have got a better network setup that would work through the VPN.

This was a good tip! I have a a situation that requires me to use two separate vpns, one being OpenVPN tap tunnel and the other is a Netscaler Gateway Plugin. The routing out of WSL2 works for the OpenVPN connection but could not find a working solution for the Netscaler based vpn... Containers running on the new WSL2 integration from Docker work flawlessly with either vpn network. For my usecase this is a good workaround as I only really need connection to the Netscaler network from the container.

Looking forward to full resolution on this for WSL2 but this helped me!

Luis-Palacios commented 4 years ago

I see workarounds for all kinds of VPN providers over here but no one has mentioned express VPN, I use their app and connect correctly:

image

On WSL 2 Ubuntu 20 I can

image

But If do something like sudo apt-get update I get:

image

Also If my app running on WSL 2 make an http request to an external api I get:

(Caused by SSLError(SSLError(1, '[SSL: WRONG_SIGNATURE_TYPE] wrong signature type

Can anyone suggest a workaround for using express vpn other than changing VPN provider 😢

Update I was able to work around by creating a VPN Network using the VPN built-in in windows for Express VPN using the following instructions: https://www.expressvpn.com/support/vpn-setup/manual-config-for-windows-10-with-l2tp/

And then doing in PowerShell

netsh.exe interface ipv4 show interfaces

So that I can see the MTU from the VPN network and then set it in WSL 2 with sudo ifconfig eth0 mtu 1200

The WRONG_SIGNATURE_TYPE error was unrelated to the VPN stuff

baruchiro commented 4 years ago

Without any change (I think) somethink starts to work.

I'm connected to the Checkpoint VPN, and here some ping outputs:

$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=118 time=72.0 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=118 time=63.6 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=118 time=66.8 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=118 time=59.6 ms
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 59.643/65.519/71.989/4.517 ms

internal IP:

$ ping 10.45.0.16
PING 10.45.0.16 (10.45.0.16) 56(84) bytes of data.
64 bytes from 10.45.0.16: icmp_seq=1 ttl=62 time=91.3 ms
64 bytes from 10.45.0.16: icmp_seq=2 ttl=62 time=90.9 ms
64 bytes from 10.45.0.16: icmp_seq=3 ttl=62 time=90.5 ms
64 bytes from 10.45.0.16: icmp_seq=4 ttl=62 time=99.3 ms
^C
--- 10.45.0.16 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 90.496/92.987/99.277/3.641 ms
$ ping google.com
PING google.com (216.58.211.206) 56(84) bytes of data.
64 bytes from mrs09s11-in-f14.1e100.net (216.58.211.206): icmp_seq=1 ttl=118 time=64.0 ms
64 bytes from mrs09s11-in-f14.1e100.net (216.58.211.206): icmp_seq=2 ttl=118 time=59.1 ms
64 bytes from mrs09s11-in-f14.1e100.net (216.58.211.206): icmp_seq=3 ttl=118 time=66.6 ms
64 bytes from mrs09s11-in-f14.1e100.net (216.58.211.206): icmp_seq=4 ttl=118 time=87.5 ms
^C
--- google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3006ms
rtt min/avg/max/mdev = 59.070/69.281/87.463/10.838 ms
$ sudo apt update
Err:1 http://archive.ubuntu.com/ubuntu focal InRelease
  Temporary failure resolving 'archive.ubuntu.com'
Err:2 http://security.ubuntu.com/ubuntu focal-security InRelease
  Temporary failure resolving 'security.ubuntu.com'
Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done
22 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal/InRelease  Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/focal-security/InRelease  Temporary failure resolving 'security.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones used instead.
$ ping archive.ubuntu.com
PING archive.ubuntu.com (91.189.88.152) 56(84) bytes of data.
64 bytes from actiontoad.canonical.com (91.189.88.152): icmp_seq=1 ttl=55 time=83.5 ms
64 bytes from actiontoad.canonical.com (91.189.88.152): icmp_seq=2 ttl=55 time=82.8 ms
64 bytes from actiontoad.canonical.com (91.189.88.152): icmp_seq=3 ttl=55 time=83.5 ms
64 bytes from actiontoad.canonical.com (91.189.88.152): icmp_seq=4 ttl=55 time=83.4 ms
^C
--- archive.ubuntu.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 82.815/83.296/83.537/0.286 ms

Why it can't pull updates from archive.ubuntu.com but can ping to?

opensiriusfox commented 4 years ago
Get-NetIPInterface -InterfaceAlias "vEthernet (WSL)" | Set-NetIPInterface -InterfaceMetric 1
$VPNInterfaceName = "Ethernet 3"
Get-NetIPInterface -InterfaceAlias $VPNInterfaceName | Set-NetIPInterface -InterfaceMetric 5001

I'm on the GlobalProtect VPN, and the above works, but it disables internal company routing domains. Still working how to fix that bit.

ndrsg commented 4 years ago

How it worked for me, maybe someone understands whats happening with this infos:

  1. Reboot
  2. Check "Control Panel\Network and Internet\Network Connections", that it does not show "vEthernet (WSL)
  3. Connect VPN (in my Case Checkpoint)
  4. Start WSL2 -> in Windows the Network Adapter "vEthernet (WSL)" is created image

-> at this point i had basic connectivity (can ping VPN internal ips, wget / curl with http, but no https)

  1. set MTU inside WSL to the value of your VPN, (1350 for me) -> now https is working, but still i cannot apt update ("temporary errror in resolving..")

  2. lookup company nameservers in windows (details of VPN Connection under DNS-Servers)

  3. add those DNS servers to /etc/resolve.conf image