microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.24k stars 811 forks source link

wsl 2 ubuntu 18.04 unable to connect to IP resources with Cisco Annyconnect active #4277

Open eclay11 opened 5 years ago

eclay11 commented 5 years ago

Please fill out the below information:

When I start the Ubuntuu 18.04 instance I'm able to communicate with resources using IPs and hostnames.

eclay@wh-lpt-eclay:$ ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=49 time=115 ms 64 bytes from 8.8.8.8: icmp_seq=2 ttl=49 time=95.4 ms 64 bytes from 8.8.8.8: icmp_seq=3 ttl=49 time=77.3 ms 64 bytes from 8.8.8.8: icmp_seq=4 ttl=49 time=92.6 ms 64 bytes from 8.8.8.8: icmp_seq=5 ttl=49 time=90.6 ms 64 bytes from 8.8.8.8: icmp_seq=6 ttl=49 time=78.9 ms 64 bytes from 8.8.8.8: icmp_seq=7 ttl=49 time=84.1 ms 64 bytes from 8.8.8.8: icmp_seq=8 ttl=49 time=90.9 ms 64 bytes from 8.8.8.8: icmp_seq=9 ttl=49 time=80.7 ms 64 bytes from 8.8.8.8: icmp_seq=10 ttl=49 time=80.0 ms ^C --- 8.8.8.8 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 9081ms rtt min/avg/max/mdev = 77.316/88.602/115.084/10.734 ms eclay@wh-lpt-eclay:~$ trace tracepath traceroute6 traceroute6.iputils tracerpt.exe eclay@wh-lpt-eclay:$ tracepath 8.8.8.8 1?: [LOCALHOST] pmtu 1500 1: wh-lpt-eclay.mshome.net 0.497ms 1: wh-lpt-eclay.mshome.net 0.379ms 2: 10.179.10.1 2.890ms 3: 192.168.1.1 8.474ms 4: 192.168.1.1 10.395ms pmtu 1430 4: 172.26.96.161 71.223ms 5: 172.18.84.36 73.898ms asymm 6 6: 12.249.2.25 73.083ms asymm 7 7: 12.122.146.97 96.016ms asymm 14 8: 12.122.111.33 95.857ms asymm 13 9: sffca22crs.ip.att.net 95.209ms asymm 12 10: 12.122.163.61 90.288ms asymm 11 11: 12.255.10.236 103.555ms asymm 15 12: no reply 13: no reply 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 1430 Resume: pmtu 1430

Once I connect to my work VPN using cisco annyconnect client v 4.7.00136 I'm not longer able to connect to IP resources.

eclay@wh-lpt-eclay:$ ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. ^C --- 8.8.8.8 ping statistics --- 8 packets transmitted, 0 received, 100% packet loss, time 7277ms

eclay@wh-lpt-eclay:$ tracepath 8.8.8.8 1?: [LOCALHOST] pmtu 1430 1: ??? 0.362ms 1: ??? 0.364ms 2: ??? 0.392ms pmtu 1200 2: no reply 3: no reply 4: no reply 5: no reply 6: no reply 7: no reply 8: no reply 9: no reply 10: no reply 11: no reply 12: no reply 13: no reply 14: no reply 15: no reply 16: no reply 17: no reply 18: no reply 19: no reply 20: no reply 21: no reply 22: no reply 23: no reply 24: no reply 25: no reply 26: no reply 27: no reply 28: no reply 29: no reply 30: no reply Too many hops: pmtu 1200 Resume: pmtu 1200

ghost commented 5 years ago

Not 100% sure, but it can be that annyconnecy blocks any dns server except provided by connection. I was observed something similar and was unable to disable this behaviour.

eclay11 commented 5 years ago

I've tried the method of creating a manual named.conf and it didn't fix anything. I can ping the 1st dns server from windows but not from ubuntu. Windows DNS servers after vpn connection established.

DNS Servers . . . . . . . . . . . : 172.24.0.25 8.8.8.8 C:\Users\eclay>ping 172.24.0.25

Pinging 172.24.0.25 with 32 bytes of data: Reply from 172.24.0.25: bytes=32 time=131ms TTL=126 Reply from 172.24.0.25: bytes=32 time=103ms TTL=126 Reply from 172.24.0.25: bytes=32 time=109ms TTL=126

Ubuntu

$ ping 172.24.0.25 PING 172.24.0.25 (172.24.0.25) 56(84) bytes of data.

^C --- 172.24.0.25 ping statistics --- 190 packets transmitted, 0 received, 100% packet loss, time 196530ms

It seems to be more of a routing problem than a name resolution. For instance I can ping www.microsoft.com from a cmd prompt but not from ubunutu.

windows

ping www.microsoft.com

Pinging e13678.dspb.akamaiedge.net [23.202.232.226] with 32 bytes of data: Reply from 23.202.232.226: bytes=32 time=103ms TTL=51 Reply from 23.202.232.226: bytes=32 time=105ms TTL=51 Reply from 23.202.232.226: bytes=32 time=121ms TTL=51 Reply from 23.202.232.226: bytes=32 time=109ms TTL=51

Ping statistics for 23.202.232.226: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 103ms, Maximum = 121ms, Average = 109ms

Ubuntu

$ ping 23.202.232.226 PING 23.202.232.226 (23.202.232.226) 56(84) bytes of data. ^C --- 23.202.232.226 ping statistics --- 5 packets transmitted, 0 received, 100% packet loss, time 4176ms

WsK commented 5 years ago

Having exactly same problem...Errrrr

kobenauf commented 5 years ago

I have the same issue, that from within the WSL 2 shell, I'm unable to use AnyConnect running in Windows. However, in WSL I installed openconnect, and I run: sudo openconnect --no-dtls -u myUsername vpn.myhost.com Then I can access my company ip addresses from within WSL 2 sessions, but not from Windows, but this lets me at least ssh and run git, so that's an improvement.

Also, this only works if I disconnect the VPN in Windows first, which is annoying.

chunkaichang commented 5 years ago

Same issue here. I tried to add search list in /etc/resolv.conf followed the guide here but still failed.

craigloewen-msft commented 5 years ago

If you're still experiencing this issue could you please take some networking logs? It will help us diagnose what's going on.

You can find instructions on how to do so here!

mcurole commented 5 years ago

I'm definitely experiencing this issue and I'm running 18970. I can capture logs if needed. The problem is easy to reproduce.

mcurole commented 5 years ago

Well, cancel that. I don't have the Feedback Hub installed and I have another issue where I can't download app from the Microsoft Store because of some weird Microsoft Account issue that I'm working with support on.

MichaelTong commented 5 years ago

Also, if WSL 2 starts first, Cisco AnyConnect will fail to connect.

MichaelTong commented 5 years ago

I filed a feedback in FH. https://aka.ms/AA60j0u

cmeiklejohn commented 4 years ago

Same issue, I filed feedback using Feedback Hub.

paulguerrero commented 4 years ago

I also have the same issue with AnyConnect, also gave feedback in the Hub (although the option to post a recording was grayed out for me).

neileadobe commented 4 years ago

I'm also having this problem and opened an issue for it in Feedback Hub with logs: https://aka.ms/AA6fthe

uc-cjdavis commented 4 years ago

I have the same issue, that from within the WSL 2 shell, I'm unable to use AnyConnect running in Windows. However, in WSL I installed openconnect, and I run: sudo openconnect --no-dtls -u myUsername vpn.myhost.com Then I can access my company ip addresses from within WSL 2 sessions, but not from Windows, but this lets me at least ssh and run git, so that's an improvement.

Also, this only works if I disconnect the VPN in Windows first, which is annoying.

I tried this route, but our AnyConnect uses 2FA which makes using openconnect impossible

raisin-loaf commented 4 years ago

I'm in the same boat. WSL 2 is a godsend while I'm in the office; it's so much faster and better integrated than using a VirtualBox (or similar) VM. Unfortunately I still have to use said VM when connecting remotely via VPN because, as others here have described, all network connectivity from WSL 2 stops the moment the VPN is up.

I use Cisco AnyConnect too although I imagine the problem is common to most VPN clients.

Like @haselton I'm unable to use OpenConnect as the company I work for enforces 2FA.

@craigloewen-msft - I'd dearly love to send you some logs but our workstation diagnostic data settings are locked down by Group Policy. If there's anything else I can do to help please let me know.

WSL 2 is fabulous. 🙂

craigloewen-msft commented 4 years ago

@raisin-loaf thank you for the offer! I'll let you know if you can help out in any other way. We are investigating issues related to the VPN.

AdonisLL commented 4 years ago

I was able to resolve this by installing anyconnect pwa from the windows store the vpn connection now works with WSL when my stand alone installation of anyconnect 4.6 does not work... hope this helps someone

raisin-loaf commented 4 years ago

I was able to resolve this by installing anyconnect pwa from the windows store

It works! Thanks so much for posting this.

dalgibbard commented 4 years ago

I was having the same issue; i found that Anyconnect was setup to do full-tunnel, and therefore a route existed to throw all traffic through the VPN connection (likely including packets destined for the WSL2 vm).

WSL2 routes are configured with a metric of 5256, and anyconnect routes have a metric of 2, and therefore take priority. I amended the routes for anyconnect (where metric == 2) to be 5257 instead, which seems to largely work. I used "NetRouteView" run as Admin to do the changes because I was too lazy to find the powershell equivilent commands :)

This restored connectivity, but I found DNS to still be broken with the WSL default resolvconf (nameserver 172.xx.xx.x) so i dropped in nameserver 1.1.1.1 instead which seems OK. Will likely replace this with the work DNS for work resources to resolve nicely :) and now full connectivity is working; no additional changes needed; but this likely will need applying each time I start anyconnect.

chrisjones-brack3t commented 4 years ago

Any movement on a fix for this? In my attempt to move away from macOS to Windows for dev, this has effectively stopped me completely as the company I work for uses AnyConnect.

craigloewen-msft commented 4 years ago

Yes we are tracking this and are looking into solutions with the networking team! I'll post any updates here as soon as they become available.

caal-15 commented 4 years ago

Thanks a lot, this would be a life saver!

kzeitz commented 4 years ago

For what it's worth I was having all the same issues described here.

I saw AdonisLL post and tried it. I installed the AnyConnect UWP version available on the Windows store. Now, it seems to work perfectly! No jacking around with resolv.conf, or turning off generateResolvCon in wsl.conf.

In fact the VPN seems much faster than the AnyConnect client 4.5 version I was using. Considering all the grief this was causing, this turned out to be a fast and easy fix for this!

chrisjones-brack3t commented 4 years ago

@kzeitz Company I work for requires 2FA authentication which it seems the Windows store version of AnyConnect doesn’t support.

kzeitz commented 4 years ago

Ah, bummer. It's been so nice for this to just work likes its suppose to.

Sorry.

On Sat, Jan 18, 2020 at 7:30 PM Chris Jones notifications@github.com wrote:

@kzeitz https://github.com/kzeitz Company I work for requires 2FA authentication which it seems the Windows store version of AnyConnect doesn’t support.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/microsoft/WSL/issues/4277?email_source=notifications&email_token=ABCZVFI75HZHHGA3U3ZUCGTQ6O3NDA5CNFSM4H6UGPDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJKHCZQ#issuecomment-575959398, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCZVFM32JGHKGJP45AKXPTQ6O3NDANCNFSM4H6UGPDA .

stuardcg commented 4 years ago

No updates for this issue? I think this is related to #4246 and neither has a real solution. I use the Microsoft Store AnyConnect and it works to me but not for everyone.

chrisjones-brack3t commented 4 years ago

An update on this would be great but an actual fix would be +1000.

Atheuz commented 4 years ago

I have this problem as well, but I have a workaround, which is to adjust my /etc/resolve.conf file:

Before adjustment, when connected to VPN (with mocked ip addresses):

nameserver 111.111.111.112
nameserver 111.111.111.111
nameserver 123.123.123.123
nameserver fff0:0:0:ffff::1

After adjustment:

nameserver 123.123.123.123
nameserver 111.111.111.112
nameserver 111.111.111.111
nameserver fff0:0:0:ffff::1

I hope this workaround can work for others.

dalgibbard commented 4 years ago

@Atheuz thanks for the tip; but this isn't a DNS issue, that's a different known issue. The issue discussed here is that, on Anyconnect full tunnel clients, the routing sets a high priority metric for the interface which steals all the traffic. WSL should use a high interface metric too, or networking should allow fallback to NAT etc.

pixelandpen commented 4 years ago

I was able to resolve this by installing anyconnect pwa from the windows store the vpn connection now works with WSL when my stand alone installation of anyconnect 4.6 does not work... hope this helps someone

Can confirm this worked for me too.

edperry commented 4 years ago

I was able to resolve this by installing anyconnect pwa from the windows store the vpn connection now works with WSL when my stand alone installation of anyconnect 4.6 does not work... hope this helps someone

Can confirm this worked for me too.

I also confirmed the Anyconnect client install does not work but the one from the MS store does

jay1648 commented 4 years ago

I was able to resolve this by installing anyconnect pwa from the windows store the vpn connection now works with WSL when my stand alone installation of anyconnect 4.6 does not work... hope this helps someone

Anyconnect client from Windows store works for me too, while Cisco Anyconnect client doesn't

eperry commented 4 years ago

I am having the same problem too. Anyconnect from the App store works but is unstable/sucks and the Anyconnect package install, breaks the network in bash as soon as I successful connect

I have tested OpenConnect Gui and that seems to be an ok replacement. https://github.com/openconnect/openconnect-gui/releases

I got my choices of profiles, my 2FA and WSL2 seems to be working so this is my solution. as I can't download the most recent Cisco AnyConnect so stuck on AnyConnect 4.5.0058

HanchengWu commented 4 years ago

I was able to resolve this by installing anyconnect pwa from the windows store the vpn connection now works with WSL when my stand alone installation of anyconnect 4.6 does not work... hope this helps someone

omg, day-saver, this works with WSL2 and latest win 10 build

mwidmann commented 4 years ago

There seems to be an issue with the Anyconnect client from the Windows Store which capped the connection to 5/5 Mbit on a 250/50 line. I wasn't able to find any ways of configuring this from inside the app or the Windows VPN settings.

I then tried OpenConnect Gui suggested by @eperry and now at least I get 40/40. The regular Anyconnect client does faster connections 100/50, but doesn't work with WSL2. For now the 40/40 will work just fine, thankfully.

Would be great if this got solved soon.

fredAlm commented 4 years ago

Hi, we are also looking into start using WSL2 at my company, but this problem stops us from doing so. We need to be able to use the 2FA AnyConnect Client. Any updates on progress etc?

HanchengWu commented 4 years ago

Using any connect from Microsoft store might solve the problem

chrisjones-brack3t commented 4 years ago

@HanchengWu as stated before, the MS Store version does not work with 2FA so it is not a solution.

HanchengWu commented 4 years ago

@cmeiklejohn Hi, I am not sure which 2FA method you are using. I am using DUO with Cisco Anyconnect, and it's working perfect. For the second password, you either need to fill "push" or the authentication code from Duo app (refresh the code first). 1 2

olingerc commented 4 years ago

I guess we will all be stuck at home for more time in the coming months and VPN will be our daily bread. This has really become an issue for all those that use AnyConnect with a form of 2FA that does not work with the Windows Store App. At least 20 people in my team. We have a system with a Certificate installed on a card with a PIN needed to unlock the card. Only the old school client works there. I am now constantly changing routes which is becoming very tedious. Any progress on this or maybe even an ETA?

fredAlm commented 4 years ago

Using any connect from Microsoft store might solve the problem

We need to use the 2FA version (store version isn't supporting this).

dalgibbard commented 4 years ago

Judging by this: https://www.networkworld.com/article/3543298/cisco-others-shine-a-light-on-vpn-split-tunneling.html We might see split-tunnel support from several providers (Cisco Anyconnect included) soon.

I guess it might still boil down to configuration and implementation, but it's a potentially positive note regardless.

YoungElPaso commented 4 years ago

I was having the same issue; i found that Anyconnect was setup to do full-tunnel, and therefore a route existed to throw all traffic through the VPN connection (likely including packets destined for the WSL2 vm).

WSL2 routes are configured with a metric of 5256, and anyconnect routes have a metric of 2, and therefore take priority. I amended the routes for anyconnect (where metric == 2) to be 5257 instead, which seems to largely work. I used "NetRouteView" run as Admin to do the changes because I was too lazy to find the powershell equivilent commands :)

...

This restored connectivity, but I found DNS to still be broken with the WSL default resolvconf (nameserver 172.21.48.1) so i dropped in nameserver 1.1.1.1 instead which seems OK. Will likely replace this with the work DNS for work resources to resolve nicely :) and now full connectivity is working; no additional changes needed; but this likely will need applying each time I start anyconnect.

So I was having exactly this issue and this worked perfectly for me. I also had to change nameserver to our VPN DNS but that was easy. Changing metric was the key element!💡 I should note, we use 2fa so using the MS Store Anyconnect client didn't work, nor did trying to hack around with OpenConnect. Thanks for the insight!

flybyray commented 4 years ago

if vpn is activated find an ip which is routed through your vpn tunnel and which is pingable. choose a packet size which is not producing an error with packet fragments. ping -f -l <<packet_size>> <<some_ip_routed_via_vpn>>

e.g: from Windows PS ( -sorry german Output)

PS> ping  -f -l 1373 10.12.1.12

Ping wird ausgeführt für 10.12.1.12 mit 1373 Bytes Daten:
Paket müsste fragmentiert werden, DF-Flag ist jedoch gesetzt.
Paket müsste fragmentiert werden, DF-Flag ist jedoch gesetzt.

than check again and decrease the packet size

PS> ping  -f -l 1372 10.12.1.12

Ping wird ausgeführt für 10.12.1.12 mit 1372 Bytes Daten:
Antwort von 10.12.1.12: Bytes=1372 Zeit=29ms TTL=62
Antwort von 10.12.1.12: Bytes=1372 Zeit=29ms TTL=62

Go into wsl/2 and set this mtu on the eth0 interface

$ sudo ifconfig eth0 mtu 1372

without this most https handshakes and other networking is broken.

Lc0rE commented 4 years ago

I don't know if it could help someone. but I got it working with the OpenConnect VPN client: https://github.com/openconnect/openconnect-gui/releases

drehelis commented 4 years ago

One way to workaround this is for me was:

cat << EOF | sudo tee -a /etc/wsl.conf  
[network]
generateResolvConf = false
EOF
echo "nameserver <CORP DNS SERVER>" > /etc/resolv.conf

and... in elevated windows every single time I connect to to VPN 😢 (fortunately, not very often)

Get-NetAdapter | Where-Object {$_.InterfaceDescription -Match "Cisco AnyConnect"} | Set-NetIPInterface -InterfaceMetric 6000
nippyin commented 4 years ago

I was able to resolve this by installing anyconnect pwa from the windows store the vpn connection now works with WSL when my stand alone installation of anyconnect 4.6 does not work... hope this helps someone

Yes it works.

shelper commented 4 years ago

I have the same issue as the title says, and I tried pwa annyconnect but i have the error below saying workstation did not match a dynamic access policy...

Snipaste_2020-06-02_22-17-53

KinIcy commented 4 years ago

One way to workaround this is for me was:

cat << EOF | sudo tee -a /etc/wsl.conf  
[network]
generateResolvConf = false
EOF
echo "nameserver <CORP DNS SERVER>" > /etc/resolv.conf

and... in elevated windows every single time I connect to to VPN 😢 (fortunately, not very often)

Get-NetAdapter | Where-Object {$_.InterfaceDescription -Match "Cisco AnyConnect"} | Set-NetIPInterface -InterfaceMetric 6000

this worked for me, except that I had to do:

echo "nameserver <CORP DNS SERVER>" > /run/resolvconf/resolv.conf

instead

nippyin commented 4 years ago

Jason, I would rather simply install Cisco AnyConnect from Microsoft Store. it simple works without any hassle.