Networking issues while using VPN

[esabelhaus]

I've tried approaching this two different ways.

Create VPN Within Windows

• Instantiate VPN tunnel using AnyConnect VPN client on my Windows laptop, then connect to upstream devices using SSH via Linux subsystem.
• RESULT: DNS was not properly handed off to the linux subsystem, and no hostname resolution is possible.

  Create VPN Connection Within Subsystem

• Instantiate VPN tunnel using OpenConnect networking client
• RESULT: Failed to instantiate network tunnel with the following error mkdir: cannot create directory ‘/dev/net’: No such file or directory

Any help would be greatly appreciated, as I often perform work on VMs which are located behind a firewall of some sort

[tastle73]

Same thing with OpenVPN

[esabelhaus]

For the time being, it is possible to manually configure host names in /etc/hosts, but long run this is not a reliable solution unfortunately.

ex:

127.0.0.1    localhost.localdomain
192.168.1.1  my.host.name
192.168.1.2  some.other.host.name

[saschagehlich]

This is an important issue and pretty much the only thing that holds me back from switching from Mac back to Windows

[sunilmut]

@esabelhaus - Thanks for providing this feedback. Regarding "Create VPN Within Windows" scenario, were the right entries not getting populated in /etc/hosts automatically?

Regarding "VPN Connection Within Subsystem", unfortunately, WSL currently does not support '/dev/net'. If that is an important scenario for you (and @saschagehlich), please help us prioritize that providing that feedback through our User Voice Page.

[esabelhaus]

specifically, when the VPN is running on my windows machine, the linux subsystem was not properly handling host name resolution, while my windows machine was. It would appear to be DNS related at that point.

This symptom occurs both when starting the subsystem before or after I have instantiated my VPN connection. IP addresses do resolve, but the host name resolution eg /etc/hosts does not properly handle the new dns providers which are passed in from the VPN device on windows

[jwsloan]

@esabelhaus, am I understanding correctly that you were able to get around this issue? If so, could you please elaborate on your fix for someone who isn't as familiar with /etc/hosts and the like?

[esabelhaus]

Here is a decent reference for how /etc/hosts works. http://www.tldp.org/LDP/solrhe/Securing-Optimizing-Linux-RH-Edition-v1.3/chap9sec95.html

The gist is, you're mapping an IP address to a hostname, and at startup, the Ubuntu based linux subsystem is reading in the configuration file /etc/hosts and using that information to map networking to a hostname. Everything must be routed over TCP/IP, so we're telling the linux host that:

10.0.0.2    my.host.name

would allow me to ssh into 10.0.0.2 using the hostname my.host.name using the command ssh user@my.host.name

You can sort of understand why this is not a reasonable workaround, because it requires you to manually configure your host configuration for any hostname which you must access within the VPN subnet.

Hope that explanation helps!

-Haus

[jwsloan]

That definitely helps! Not too complicated at all.

Unfortunately, that didn't fix my issue.

I'm using ruby and net/http to send a POST request to a URL behind the VPN.

While connected to VPN, I used nslookup to find the IP of the host I'm trying to connect to. I edited /etc/hosts to include that IP and host. I then run my ruby code (which works through the windows command line). I get a 404 response in WSL.

Do you see something I might be missing?

[esabelhaus]

try running dig my.host.name on your host, and see whether the IP address is what you have referenced in your /etc/hosts config

Also, if you've got access to the host behind the firewall directly, maybe tail the logs server side to see whether the resource is actually available, going off the 404 error.

[aseering]

For what it's worth, a 404 error means that you did successfully connect to the server in question (which is all that /etc/hosts has control over) but that host is reporting that the particular page that you requested doesn't exist.

Is it possible that you made a typo in the hostname? Or otherwise didn't type it exactly as it appears / is used (including prefixes like "www.", etc). If you did, and if your remote server is using SNI, this would cause it to route you incorrectly.

You could also try using curl or wget, or installing VcXsrv and running firefox, to browse the server manually from within WSL.

[jwsloan]

Your instructions for installing VcXsrv are fantastic! Thank you for that. I am able to run firefox, but it gets a 404 page as well. I get the same 404 page whether I am connected to VPN or not.

I take the same URL and paste it into Chrome running in Windows, and the page works.

Any other thoughts? Thank you very much for your time, by the way!

[jwsloan]

@aseering, you were right all along. Firefox from WSL was a big help figuring it out.

In the end it was pretty simple. I put the wrong IP address in /etc/hosts. It was a public URL and not the 192.168... IP. Once I fixed that, I was able to make the connection. Thanks, and I can't wait for @bitcrazed and team to get this particular kink worked out!

[ericblade]

One thing you can do, if the software you need is capable of operating entirely through a proxy server, is setup a ssh tunnel with proxy on the host, and then use that. I can use that with Firefox in WSL to connect to sites on my work network .. I have a ssh tunnel setup on localhost that goes through another machine which is actually connected to VPN.

The problem that I have, is that I can't connect with anything but firefox, since there's no real way to force Linux to connect to everything through a SOCKS5 Proxy (unlike with Windows and Mac, there are several softwares out there that make that happen) ... as far as I've ever been able to find ... so every individual piece of software needs to be configured to use it.

So, the configuration that I use:

One (Linux) machine connected to VPN (also handles a host of other tasks, but it's basically a server box -- sits in a closet, and is only operated via remote). My primary Windows computer. Windows computer has a program called MobaXTerm installed, which provides ssh, tunnel, proxy, and X server (among many other things!). Windows also has a program called Proxifier that routes literally everything destined for a host on the other side of the VPN through a ssh tunnel socks5 proxy. Configure a ssh tunnel/socks5 proxy in MobaXTerm to go through the Linux box. Works great in Windows. In WSL, configure Firefox to use the proxy. That also works great, but also sends all traffic through the work VPN, which is not really acceptable, and is part of why I use this configuration.

The part that confuses me, is that WSL is completely unable to see the other side of the VPN tunnel. (Coincidentally, neither is Microsoft Edge -- both seem to bypass the DLL injection that Proxifier does, which makes them kind of bad citizens .. although I can justify it more in my head for WSL than for Edge..)

[rodrymbo]

One primary problem reported by the OP (@esabelhaus) is that DNS changes in Windows (e.g. by a VPN) do not get reflected to the /etc/resolv.conf in real time on WSL. Add to this that WINS resolution isn't configured into WSL at all by default (currently).

Even to veteran linux users, when DNS isn't working, it "feels" like IP networking has failed, and that's how it frequently gets reported, when just fixing DNS would get things going again. Since Windows is providing the routing table and retworking devices (transparently), having /etc/resolv.conf set up once per session, and with the DNS resolver settings of just one network adapter (I think), leads to many situations where DNS fails for WSL and with it the "feeling" that networking is broken.

Here's a suggestion: set up the Windows resolver to allow WSL to use it (connecting via UDP and TCP), so /etc/resolv.conf can list localhost as the resolver. That way all of the networking (devices, pseudo-devices, routing, DNS, etc) would be going through Windows, and problems in WSL would show up as problems in Windows, and fixing them in Windows via Windows tools would resolve them in WSL.

I'd expect that to be a much nicer UX.

Setting things up so WSL is handling all networking (interfaces, iptables/firewall/routing, DNS, VPN, tun/tap, etc; perhaps even dhcp and vpn protocols) is flexible but fairly complex, and, likely, everything on the Windows side would have to be replicated to the WSL side. So I wouldn't be disappointed if the current philosophy of "passing networking off to Windows" was maintained. Letting the local Windows do DNS resolution via /etc/resolv.conf would be straightforward and a "linuxy" way to handle DNS, at least from a user's point of view, and wouldn't require trying to keep /etc/resolv.conf (or /etc/hosts) up to date in real time as things change on the Windows side.

[ericblade]

Sounds like a good idea. DNS isn't the root of my problem, unfortunately, but I'd welcome the improvements.

[t-money-g]

@aseering That's pretty wild ... X Windows on Windows.

[ewan-fanduel]

Still having the same issue as @esabelhaus (subsystem ignoring any Windows VPN connection), has there been any update on this?

[jwsloan]

@ewan-fanduel have you tried manually editing your /etc/hosts file?

[ewan-fanduel]

@jwsloan I can yep, but the hosts I work with change often so it's a bit of hassle to keep updating it manually :/

[t-money-g]

@ewan-fanduel that sucks, does your router support dynamic DNS[https://en.wikipedia.org/wiki/Dynamic_DNS]?

[ericblade]

editing hosts doesn't work for my situation, because i'm routing the traffic destined for VPN through a ssh tunnel to end up going through another computer that is connected to the VPN directly. WSL traffic completely ignores the routing performed by the Proxifier app in Windows. (so does Microsoft Edge, also .. these are things that shouldn't be possible, IMO)

[matthiassb]

Just to toss in some information.

This issue arises for me when split tunneling is enabled. It looks like WSL uses the nameservers related to the default route with the lowest metric.

I changed the metric for the default route of the VPN to make it lower than the one for my LAN. After doing this the nameservers in /etc/resolv.conf switched orders with the ones related to the VPN interfaces at the top. However of course due to split tunnelling I can no longer access the public net with this configuration.

Win10 doesn't suffer from this issue. Even with a higher metric on the default route Windows still prioritizes using the VPN connections DNS servers.

[sunilmut]

@matthiassb - Thanks for reporting the issue. We are aware of our shortcomings in supporting VPN, and are looking into it. Meanwhile, if you are familiar with what needs to be enabled on the Linux user-mode (resolver daemons?) to better support VPN, do let us know.

[matthiassb]

@sunilmut - I'm not sure what can be done via Linux user-mode. Perhaps using something like dnsmasq could help but I see that needing to be configured per domain, etc.

I'm not sure how resolvconf get's passed the IPs from the windows space. I do see that the following PS command accurately represents the order of the DNS servers.

Get-DnsClientServerAddress -AddressFamily IPv4 | Select-Object -ExpandPropert ServerAddresses

Perhaps the underlying mechanism for the above command can be passed to resolvconf or whatever daemon is managing the DNS system in WSL

[bitcrazed]

@sunilmut While you're looking into this, could you please explore supporting DirectAccess too?

[misenesi]

@esabelhaus and @tastle73 could you please provide output when you do

ipconfig /all

?

I have a fix prepared for DNS resolution when you connect to VPN on windows, but need to verify that your VPN networking interface is reported as point-to-point interface (which is a Windows requirement).

[misenesi]

I have checked in a fix that will address some of the VPN DNS resolving issues. Currently for DNS resolving we update /etc/resolv.conf from the service. In this fix I added the capability to manually modify the /etc/resolv.conf file if you wish so, disabling its automatic regeneration.

However, due to how DNS resolving works for various VPN solutions, this fix will only work with strict force tunnel VPN that do not hide their DNS servers for privacy or security reasons.

I have a proposal for better solution that is currently under discussion that would work for all VPN scenarios.

[sunilmut]

There should be better support for some VPN solutions in 15002. Please validate how your VPN behaves with 15002 and let us know what works, doesn't work etc.

[ericblade]

1- which changes specifically?

2- any idea what vpns that affects? and does that mean using them when activated in Windows, or activating them in bash ?

... is this in time to make the next Windows update, for those of us who are sticking to the really slow track? :-D

[jcq]

Just for clarification, do these changes apply only to VPNs set up through the native Windows 10 interface? I assume that's the case, as I don't see any change when using OpenVPN (which has its own Windows client).

[sunilmut]

@ericblade

1- which changes specifically?

In [15002](), see VPN DNS propagated to /etc/resolv.conf (GH #416, 1350). @misenesi has provided more details about the solution.

2- any idea what vpns that affects?

We don't have a list of VPN's that are currently supported. As @misenesi suggests above, it is a limited solution at the moment. We will take a relook at this for the next release (post Creators Update).

and does that mean using them when activated in Windows, or activating them in bash ?

This is only for Windows VPN solutions. Linux VPN solutions are currently not supported.

is this in time to make the next Windows update, for those of us who are sticking to the really slow track?

Yes, these changes will make it to the Creators Update. If you rather prefer to stay on the slow track, you have the option to validate things on a VM.

[sunilmut]

@jcq - Yes, this only applies to Windows VPN solutions. Linux VPN solutions are currently not supported. There are open GitHub issues for that and also user voice ticket, if you search around.

[ericblade]

A nice improvement, will make it at least possible for me to do my work that requires bash, if i login to the VPN from this machine.. i prefer using my setup where my traffic is all forwarded via a hook into WinSock though .. but thank you all for your work on it :)

[regisbsb]

I'm using anyconnect as well. As soon as I connect to the vpn the DNS resolution stops even though it's using the same dns server.

[EvHaus]

@sunilmut I am using GlobalProtect VPN on Windows Build 15014, and my Bash terminal can't make any connections via to VPN addresses. Any info I can provide to help track this down?

[l8nite]

Experiencing the same here with AnyConnect VPN and split tunneling. I reported a very similar bug on docker a few months ago: https://github.com/docker/for-win/issues/174 - it might be worth pinging them to find out what their fix was.

Edit: Adding an options rotate to my /etc/resolv.conf seems to have helped. Edit2: Just kidding, that just made it non-deterministic failure.

[esabelhaus]

Wow, been a while since I posted back here. Sorry to have kinda dropped off the conversation. Moved my laptop back over to Ubuntu, but I should be able to set up my desktop PC to test this out again.

If I get it rolling, I'll see if I can get any metrics to help. I've tried the approach with Dnsmasq previously on a VM, it was kinda successful, but touch an go.

I would think the only valid solution here is that windows provides a light weight DNS server on the host which provides all known nameserver listings seen from the host perspective, but that sounds a bit excessive. In this solution, Windows would know all routing tables of all connections, and could provide a single source of DNS to the Bash subsystem

[syscrusher1709]

Hello, all.

I am a Linux veteran but very new to Windows 10 and certainly to WSL. I appreciate very much that Microsoft is incorporating smooth coexistence with Linux.

This problem replicates with the current public version of Windows 10 Pro and WSL, and with the current version of OpenVPN. (You probably assumed OpenVPN was affected, but I wanted to confirm specifically that it is, and not just IPsec.)

I would give a +1 to the suggestion of letting the Windows host be available as a DNS resolver or DNS proxy for the Linux environment. Ideally, this would be a default behavior that could be overridden if the Linux user really needed local resolver control (perhaps a change in /etc/nsswitch.conf would suffice).

The second alternative, preventing /etc/resolv.conf from being automatically overwritten, would work for me personally as an alternative, but it's not as general a solution.

One other way to work around the problem might be a way to restart WSL after the VPN is connected. I went into the Local Services settings and didn't see anything obviously related, but as I said I'm new to this environment. I'll do some searching, because I'm sure this is already documented somewhere. :)

[mikchaos]

I was having the same issue that i couldn't open ssh connections to hosts within the company network after dialing into the VPN (Cisco AnyConnect).

Here's what I found:

• Although ssh user@hostname came up with an error "ssh: Could not resolve hostname *****: Name or service not known", the DNS lookup worked by using the "host" command. So "host 'hostname'" gave me the IP address. Now I was able to use ssh with the IP address
• I then had a look at my resolv.conf and noticed that the new DNS from the VPN config was added at the bottom of the list. I changed it to be the first in the list and now "ssh user@hostname" works for me

Is there a way to make the subsystem cycle through all DNS servers listed in the resolv.conf?

[EvHaus]

Just thought I'd note that I'm on the creators update and still having VPN issues in Bash.

[betalb]

The same here, we are using. Possibility to disable automatic update of resolve.conf, won't help a lot, as I'm using notebook as work PC, and network changes pretty often

[sunilmut]

@EvNaverniouk (and @all) - Yes, we absolutely understand that our current DNS solution does not work well with many VPN software. We are looking into this. Meanwhile, please consider the option of manually overriding /etc/resolv.conf (though there seems to be a problem with that as reported by @betalb and in #1350). We are aware that that solution does not work very well if you are on a laptop and roaming around. Bear with us while we work on a scalable and sustainable fix for this.

[trondhindenes]

I'm still seeing this issue using Juniper Pulse VPN. Running Windows 10 build: 10.0.15063 Build 15063. (which I'm assuming is the creators update). As far as I could tell this should be fixed in my current build?

[mikchaos]

I still have the same issue with Cisco Anywhere VPN on Creators Update. Version: 1703, OS Build: 15063.138

I still have to manually edit /etc/resolv.conf and put the VPN DNS entry to the top of the list in order to resolve internal hostnames.

[rodrymbo]

Anybody got some free time? How about writing a service, running on the Windows side, that would listen for local DNS resolution requests from the WSL side and return whatever a program running on the Windows side would receive as an answer, including (if appropriate) mDNS and WINS names or whatever (without having to do separate requests for other names). Then /etc/resolv.conf could be some variation on nameserver localhost.

Seems to me that's about the only way to insure that DNS resolution on the WSL side stays in sync with DNS changes on the Windows side, and that names that people are used to resolving in Windows programs (e.g. WINS names) resolve the same way in WSL programs. Or at least it is a way that might work without manually checking/editing /etc/resolv.conf and /etc/hosts every few minutes, installing samba or avahi, or whatever. At least until the WSL wizards figure something else out.

Changing /etc/resolv.conf on WSL when DNS changes in Windows is problematic in that linux programs can't be guaranteed to read it afresh every time they want to make a name resolution; they rely on what they got when the process read it at initialization (though some check for modification from time to time and reread it). At least I seem to recall that being the case. Could be wrong...

[l8nite]

@rodrymbo Yea, this is basically what the docker-for-win guys did. They have a dns proxy service running that they funnel requests into.

[sunilmut]

@rodrymbo - Thanks for your write up. Yes, we haven't ruled that option out and it is looking more and more like the direction we are aligning towards too. But, we haven't finalized anything yet.

@l8nite - Do you have any references/web-links to what you are referring to as "dnx proxy service"?

[l8nite]

@sunilmut Sort of. You can read through the issue I linked in my earlier reply to this thread: https://github.com/docker/for-win/issues/174 - some of the issues that were closed as a duplicate of that also contain scattered information. Your best bet might be to reach out to one of the contributors there and ask directly. I haven't done any digging to see if their implementation is available in their repository either.

[sunilmut]

@l8nite - Thanks for linking the git hub post. I was just curious where you got the information about the dns-proxy. I can follow up internally.

[aseering]

@rodrymbo -- the service that you describe, apparently already exists, and is discussed here: https://github.com/Microsoft/BashOnWindows/issues/1815#issuecomment-289625106