Internal Host DNS cannot be resolved (Windows Container 1803)

[n-junge]

Expected behavior

According to the Docs host.docker.internal resolves the host's ip.

Actual behavior

Ping request could not find host host.docker.internal Network works fine. Pinging Host IP directly (ipconfig) works as expected.

Host: Windows 10 Enterprise Container: microsoft/windowsservercore:latest

Neither stable (18.03) nor edge (18.04) works.

Probably important side note: DNS can be resolved in Linux Container (Ubuntu)

 PS C:\> ping host.docker.internal
 Ping request could not find host host.docker.internal. Please check the name and try again.
 PS C:\> ping gateway.docker.internal
 Ping request could not find host gateway.docker.internal. Please check the name and try again.
 PS C:\> ping google.com

 Pinging google.com [172.217.16.78] with 32 bytes of data:
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46

 Ping statistics for 172.217.16.78:
     Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
 Approximate round trip times in milli-seconds:
     Minimum = 31ms, Maximum = 31ms, Average = 31ms

C:\WINDOWS\system32>docker version
Client:
 Version:       18.03.0-ce
 API version:   1.37
 Go version:    go1.9.4
 Git commit:    0520e24
 Built: Wed Mar 21 23:06:28 2018
 OS/Arch:       windows/amd64
 Experimental:  false
 Orchestrator:  swarm

Server:
 Engine:
  Version:      18.03.0-ce
  API version:  1.37 (minimum version 1.24)
  Go version:   go1.9.4
  Git commit:   0520e24
  Built:        Wed Mar 21 23:21:06 2018
  OS/Arch:      windows/amd64
  Experimental: false

[jasonbivins]

Hi @4xiz Thanks for reporting - I can repro and have reported it as a bug

[ebriney]

It's a windows daemon limitation and needs to be fix there but we will add a workaround waiting MS to fix it upstream.

[n-junge]

Release 18.05.0-ce-win66 works. Thanks.

[atomaras]

Tested it on Windows 10, Docker 18.05.0-ce-win66 (17760) and it doesn't work

[n-junge]

@atomaras Well the dns resolution seems to work for you too. So that's a different problem. I propose you open another issue.

[JoshCollinsMSFT]

I just tried this and it isn't working for me. Version info:

Windows Microsoft Windows [Version 10.0.17134.112]

Docker

Client:
 Version:      18.05.0-ce
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   f150324
 Built:        Wed May  9 22:12:05 2018
 OS/Arch:      windows/amd64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      18.05.0-ce
  API version:  1.37 (minimum version 1.24)
  Go version:   go1.10.1
  Git commit:   f150324
  Built:        Wed May  9 22:29:00 2018
  OS/Arch:      windows/amd64
  Experimental: false

Ping output:

C:\>docker exec b61746027bc9 ping host.docker.internal
Ping request could not find host host.docker.internal. Please check the name and try again.

[cpumanaz]

@JoshCollinsMSFT I have the same windows build (17134.112) and docker (18.05.0-ce) and am also not able to resolve host.docker.internal. No joy with connection strings.

[prochnowc]

Still not working for me with 18.06.0-ce:

docker run --rm -i microsoft/nanoserver:1709 ping host.docker.internal
Ping request could not find host host.docker.internal. Please check the name and try again.

[Heurazio]

Same problem. docker.host.internal not working.

[prochnowc]

@n-junge @ebriney could you please re-open this issue?

[kierzo]

So it seems the way this was fixed is that it must write out to the local hosts file on the host machine.

I have a bit of a situation as Symantec Endpoint blocks write access to the hosts file :(. Is there a better dns way of doing this? Thanks

[ebriney]

They are 2 separate things.

First, for dockerd to get host.docker.internal resolution we must put it in windows hosts file... So if you have endpoint protection or similar you must add an exception on com.docker.service executable.

Second, in the windows containers, we do a docker exec as soon as the container is started (using docker event) to patch the hosts file in the container (I don't know what happened if you are running in process isolation, because we don't support Windows server). So if you overwrite it at runtime you can lose those entries.

@Heurazio this is host.docker.internal, not docker.host.internal. you also have gateway.docker.internal but that is the same ip in windows containers.

@prochnowc can you try docker run microsoft/nanoserver powershell "Start-Sleep -s 2 ; if (! $(IPCONFIG /displayDNS | Select-String -Quiet -SimpleMatch host.docker.internal)) { exit 1 }"

[prochnowc]

@ebriney Interestingly ... if i wait 2 seconds after startup i can ping host.docker.internal. I will try this again within my traefik for windows container. traefik needs the gateway address.

[prochnowc]

Unfortunately it does not work.. traefik_1 | time="2018-08-25T01:30:05+02:00" level=error msg="Failed to retrieve information of the docker client and server host: error during connect: Get http://host.docker.internal:2375/v1.24/version: dial tcp: lookup host.docker.internal: getaddrinfow: This is usually a temporary error during hostname resolution and means that the local server did not receive a response from an authoritative server."

Does the host.docker.internal stuff require minimum nanoserver version?

[ebriney]

@prochnowc, we execute that script as soon as we receive the start event from any windows containers:

$hostsFile = "C:\Windows\System32\drivers\etc\hosts"
$src = [System.IO.File]::ReadAllLines($hostsFile)
$a = $src += ""
For ($i=0; $i -le $a.length; $i++) {
    if ($a[$i].Contains("host.docker.internal"))
    {
        $a[$i] = "%s host.docker.internal"
        $a[$i+1] = "%s gateway.docker.internal"
        [System.IO.File]::WriteAllLines($hostsFile, [string[]]$a)
        exit
    }
}
$a = $a += "# Added by Docker for Windows"
$a = $a += "%s host.docker.internal"
$a = $a += "%s gateway.docker.internal"
$a = $a += "# End of section"
[System.IO.File]::WriteAllLines($hostsFile, [string[]]$a)

Perhaps the hosts file doesn't exist and there is an exception reading it.

[prochnowc]

@ebriney Am I right when saying that this requires working PowerShell in the windows container? My traefik container does not have PowerShell.

[ebriney]

@prochnowc Arfff, that's why it didn't work. Do you have a suggestion on how to solve that?

[prochnowc]

@ebriney So actually this fix only works on windows containers which have PowerShell installed (which is not the case for windows nanoserver). I dont think the hosts file editing can be done with "standard" windows command line utils. My idea would be to implement this in the docker DNS.

[ebriney]

@prochnowc hummm, nanoserver includes powershell (see command line above). And yes your image must include powershell for now. Microsoft is managing all the network part on WCOW so except if they handle it upstream we cannot do anything (but to do that I think it must be generalized in moby/moby). On Linux containers the vm traffic is going throught vpnkit so we can tweak dns requests ...

[prochnowc]

@ebriney Sorry, but nanoserver does not include PowerShell by default. See https://docs.microsoft.com/en-us/windows-server/get-started/nano-in-semi-annual-channel. Your command line didnt work with pure nanoserver image. I had to use mcr.microsoft.com/powershell:6.0.2-nanoserver.

[ebriney]

and so why docker run microsoft/nanoserver powershell "Start-Sleep -s 2 ; if (! $(IPCONFIG /displayDNS | Select-String -Quiet -SimpleMatch host.docker.internal)) { exit 1 }" is working?

[ebriney]

ok that's in 1803 image

[kierzo]

@ebriney , Would you be able to update your powershell script slightly to only write to the host file if needed?

I can see you do an if string contains "host.docker.internal" but then also add the comment strings to the file after the check by the looks of it.

Because of this second write to the file outside of this condition, this one is causing me a bit of pain because I've had my I.T update my hosts file manually to fix this, but because of this second write statement that happens unconditionally, when I click switch to Windows containers, there is a "can't write to hosts file error" still and crashes Docker.

Also in my case the wrong host ip was getting selected too as we have multiple network adaptors.

A suggestion if you could make if possible, would be to first check the DNS resolves first, if it doesn't then check if the hosts file has the additions it needs, and write to it if its not in there, also would move the second write in to the if statement.

Hope this makes sense.

I've coded and tested my suggestions above (W10) in the below, so feel free to use parts or all of it if needed: Really appreciate any help as ping ponging with my IT dept here on this which is painful.

$hostsFile = "${env:windir}\system32\drivers\etc\hosts"
# Check DNS resolves before touching the hosts file.
if((Resolve-DnsName -Name host.docker.internal) -And (Resolve-DnsName -Name gateway.docker.internal)){}
else {
    # Check if in the hosts file and write it in if its not there...
    if((cat $hostsFile | Select-String -Pattern "host.docker.internal") -And (cat $hostsFile | Select-String -Pattern "gateway.docker.internal")){}
    else {
        $ip = (Resolve-DNSName (Hostname) | Select IPAddress | select IPAddress -expandproperty IPAddress | select -last 1)
        $addtohost = "`n#Added by Docker for Windows`n$ip    host.docker.internal`n$ip    gateway.docker.internal`n# End of section`n"
        echo $addtohost | Out-File -Append -filepath $hostsFile
    }
}

[ebriney]

Ok @kierzo, thanks for the feedback, I'll fix it. And will try to re-write it in batch when I will have some time so it can work in containers without powershell (cc @prochnowc).

[kierzo]

@ebriney Excellent! thanks a lot :)

[ebriney]

@kierzo when you do $ip = (Resolve-DNSName (Hostname) | Select IPAddress | select IPAddress -expandproperty IPAddress | select -last 1) is it in the container or in the host?

[kierzo]

@ebriney , on my host, that's where I have the problem. haven't tried in a container.

I did attempt a rewrite in batch if it helps, works on my W10 test, it may help you possibly?

@echo off
SETLOCAL

SET hostsfile=%systemroot%\system32\drivers\etc\hosts

:: Get local ip
FOR /F "tokens=* USEBACKQ" %%F IN (`hostname`) DO (
    SET hostname=%%F
)
FOR /F "tokens=2 USEBACKQ delims=:" %%F IN (`nslookup %hostname%`) DO (
    SET ip=%%F
)

:: Check if DNS resolves...
ping host.docker.internal -n 1 > NUL 2>&1 && ping gateway.docker.internal -n 1 > NUL 2>&1
IF /I "%ERRORLEVEL%" NEQ "0" (
    ECHO DNS Resolution failed
) ELSE (
    :: If it resolves jump over the step to write to the hosts file...
    goto END 
)

:: Check if in hosts file or not and stick it in if its not....
TYPE "%hostsfile%" | FIND "host.docker.internal" && TYPE "%hostsfile%" | FIND "gateway.docker.internal" 
IF /I "%ERRORLEVEL%" NEQ "0" (
    ECHO. >> %hostsfile%
    ECHO %ip:~2%    host.docker.internal >> %hostsfile%
    ECHO %ip:~2%    gateway.docker.internal >> %hostsfile%
)

:END

:: Check if DNS resolves...
PING host.docker.internal -n 1 > NUL 2>&1 && PING gateway.docker.internal -n 1 > NUL 2>&1
IF /I "%ERRORLEVEL%" NEQ "0" (
    ECHO DNS still failed...
    exit /B 1
)

ENDLOCAL

[ebriney]

Thanks @kierzo, I will use your ip resolution for the host injection and the batch for the container host setup

[prochnowc]

@ebriney Should the move from PS to BAT script be in 2.0.0.0-beta1-win75? The ticket was mentioned in the release notes. Unfortunately it still does not work and I can see [08:19:57.063][WindowsDaemon ][Info ] debug: exec commandLine: powershell -Command "$hostsFile = \"C:... in the log file.

[ebriney]

@prochnowc : I only did the host part. In the container, I cannot write in hosts file, got an access denied. I need to investigate why (tested on nanoserver:1803 image).

[prochnowc]

@ebriney: In nanoserver:1803 the user is ContainerUser which does not have write access. Only ContainerAdministrator has privileged access.

[ebriney]

Hummm, ok I'll try with a -u flag but is it compatible with base nanoserver image (14586) @prochnowc ?

[prochnowc]

I dont think that -u flag is supported

[chris-sung]

shall we reopen this issue? this issue also happens with microsoft/dotnet:2.1-aspnetcore-runtime-nanoserver-1803 or microsoft/dotnet:2.1-sdk-nanoserver-1803 which is commonly used with asp.net core Dockerfile template.

[ebriney]

I change the title and re-open it @chris-sung. I found a way to make it works on 1803 using -u ContainerAdmin flag and batch script but unfortunately the same exec command freezes the daemon on old nanoserver images... I have to relaunch it which is not good at all.

[richardgavel]

The issue I'm seeing related to this is that it seems to be getting the IP address from the wrong network.

On container:

C:\>type windows\system32\drivers\etc\hosts
# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
#      102.54.94.97     rhino.acme.com          # source server
#       38.25.63.10     x.acme.com              # x client host

# localhost name resolution is handled within DNS itself.
#       127.0.0.1       localhost
#       ::1             localhost

# Added by Docker for Windows
172.26.136.33 host.docker.internal
172.26.136.33 gateway.docker.internal
# End of section

C:\>ipconfig

Windows IP Configuration

Ethernet adapter Ethernet 2:

   Connection-specific DNS Suffix  . : *****
   Link-local IPv6 Address . . . . . : fe80::f1f6:737f:2446:d4e0%5
   IPv4 Address. . . . . . . . . . . : 172.21.206.197
   Subnet Mask . . . . . . . . . . . : 255.255.240.0
   Default Gateway . . . . . . . . . : 172.21.192.1

On host:

Ethernet adapter vEthernet (nat):

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter vEthernet (Primary Network Card):

   Connection-specific DNS Suffix  . : *****.com
   Link-local IPv6 Address . . . . . : fe80::f1e7:b112:a0fd:49dd%2
   IPv4 Address. . . . . . . . . . . : 10.3.4.220
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 10.3.4.1

Wireless LAN adapter Wi-Fi:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : *****.COM

Wireless LAN adapter Local Area Connection* 2:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter Bluetooth Network Connection:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter vEthernet (Default Switch):

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::c4b6:1923:2680:2caa%9
   IPv4 Address. . . . . . . . . . . : 172.26.136.33
   Subnet Mask . . . . . . . . . . . : 255.255.255.240
   Default Gateway . . . . . . . . . :

Ethernet adapter vEthernet (nat) 2:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::b2:c004:744a:a107%4
   IPv4 Address. . . . . . . . . . . : 172.21.192.1
   Subnet Mask . . . . . . . . . . . : 255.255.240.0
   Default Gateway . . . . . . . . . :

Ethernet adapter vEthernet (c4dc5791e7e9eaa):

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::34ec:7c0c:ecf9:30bb%25
   IPv4 Address. . . . . . . . . . . : 172.18.208.1
   Subnet Mask . . . . . . . . . . . : 255.255.240.0
   Default Gateway . . .
``` . . . . . . :

Even though the container is using nat, it's not using the IP address of the host on that Ethernet card, it's using the primary. I can ping the IP address defined on the NAT network from the container, and it matches the default gateway, which it should.

[ebriney]

Hi @richardgavel, which version are you using?

[randytan]

Hi @ebriney,

I can confirm this happens as well for Docker EE- version 17.06-2-ee-16

C:\Users\Administrator\Documents>docker info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 53
Server Version: 17.06.2-ee-16
Storage Driver: windowsfilter
 Windows:
Execution Driver: <not supported>
Logging Driver: json-file
Plugins:
 Volume: local
 Network: l2bridge l2tunnel nat null overlay transparent
Kernel Version: 10.0 14393 (14393.2485.amd64fre.rs1_release.180827-1809)
Operating System: Windows Server 2016 Datacenter
OSType: windows
Architecture: x86_64
CPUs: 4
Total Memory: 16 GiB

May I know if the EE version also having the same code base as the CE version?

[kierzo]

The resolution I could think of was doing an nslookup on your computers host name e.g.

hostname nslookup myhostname

I'm wondering if this give the correct ip @richardgavel @randytan ?

[randytan]

Yes I did nslookup the hostname manually and edited the /etc/hosts inside the %Windows%/System32/Drivers/etc/hosts to include two lines of internal host addresses.

These interim solution should do the trick, but for the future releases, it should be working out of the box and the bug should be addressed by either Microsoft or Docker platform.

[ebriney]

@randytan docker ee doesn't have host.docker.internal it's a desktop feature. and is (gwmi win32_networkadapterconfiguration | where {$_.ipaddress -ne $null -and $_.defaultipgateway -ne $null} | select -First 1).IPAddress[0] gives you the right address @randytan @richardgavel @prochnowc ?

[ebriney]

even better perhaps: (gwmi win32_networkadapterconfiguration | where {$_.ipaddress -ne $null -and $_.defaultipgateway -ne $null} | Sort-Object -Property @{ Expression = "IPConnectionMetric" ; Descending = $True } | select -First 1).IPAddress[0]

[ghost]

I have a hybrid swarm with both Windows and Linux containers. From Iinux containers I can do

/ # nslookup db
nslookup: can't resolve '(null)': Name does not resolve

Name:      db
Address 1: 10.0.12.5 2c718448b234.testi_proxy-net

But from Windows ones, I cannot

PS C:\> nslookup.exe db
DNS request timed out.
    timeout was 2 seconds.
Server:  UnKnown
Address:  10.0.13.1

DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to UnKnown timed-out

My db container runs Windows Is this issue related? All my Windows containers are 1803 now (I made an upgrade recently). The Windows node runs v18.03.1-ee-3 and linux ones v18.06.1-ce. Both are latest.

[ebriney]

Hi @achrjulien , this is a separate issue, you should create a new one.

[ghost]

In case someone comes here and takes the same dodgy path. I got my problems solved by running 18.06.1-ce on both Linux (Ubuntu/Debian) and Windows (server 1803) on physical servers. For Windows, you have to compile 18.06.1-ce yourself.

[perithompson]

I'm now getting this issue on 18.09.1, I can ping the internet but host.docker.internal does not resolve. This was working before the latest update? I have tried uninstalling both hyperv and Docker but fresh install does not resolve this...

Containers: 2 Running: 2 Paused: 0 Stopped: 0 Images: 17 Server Version: 18.09.1 Storage Driver: windowsfilter (windows) lcow (linux) Windows: LCOW: Logging Driver: json-file Plugins: Volume: local Network: ics l2bridge l2tunnel nat null overlay transparent Log: awslogs etwlogs fluentd gelf json-file local logentries splunk syslog Swarm: inactive Default Isolation: hyperv Kernel Version: 10.0 17134 (17134.1.amd64fre.rs4_release.180410-1804) Operating System: Windows 10 Pro Version 1803 (OS Build 17134.523) OSType: windows Architecture: x86_64 CPUs: 8 Total Memory: 15.86GiB

Update: downgrading did not resolve this, I have tried adding DNS exceptions into the firewall and disabling, also tried RUN Set-ItemProperty -path 'HKLM:\SYSTEM\CurrentControlSet\Services\Dnscache\Parameters' -Name ServerPriorityTimeLimit -Value 0 -Type DWord ; \ New-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\Dnscache\Parameters' -Name MaxCacheTtl -Value 0 -Type DWord ; \ New-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\Dnscache\Parameters' -Name MaxNegativeCacheTtl -Value 0 -Type DWord ; \ Restart-Service dnscache

but that does not work either

nslookup, seems to reveal that it is querying the internal dns on the container nat network but gives no results, querying google works correctly...

[perithompson]

looking at this for a little longer, it seems that the only way i can get the containers to see each other using compose is by defining a hostname. Any aliases, links or service names are ignored on docker-compose up will not resolve but adding hostname: somename then allows the containers to resolve

[noraab]

Hi @perithompson , I'm facing the same problem as you do, but using your hostname workaround leads my services to resolve using ipv6 only. Have you managed getting it work using ipv4? Unfortunatly there is not option in docker for windows to disable ipv6 in general - as far as I'm aware.

[perithompson]

Hi @perithompson , I'm facing the same problem as you do, but using your hostname workaround leads my services to resolve using ipv6 only. Have you managed getting it work using ipv4? Unfortunatly there is not option in docker for windows to disable ipv6 in general - as far as I'm aware.

Hi @noraab, Yeah, it resolves on ipv6 for me too, but that hasn't caused me a problem for what I needed, I don't know if there is a way to disable it, unless you can change the settings on the hosts virtual switch?

[noraab]

To provide more details: This happens on Docker Version 18.09.1 This does not happen on Docker Version 18.09.0