Internal Host DNS cannot be resolved (Windows Container 1803)

[n-junge]

Expected behavior

According to the Docs host.docker.internal resolves the host's ip.

Actual behavior

Ping request could not find host host.docker.internal Network works fine. Pinging Host IP directly (ipconfig) works as expected.

Host: Windows 10 Enterprise Container: microsoft/windowsservercore:latest

Neither stable (18.03) nor edge (18.04) works.

Probably important side note: DNS can be resolved in Linux Container (Ubuntu)

 PS C:\> ping host.docker.internal
 Ping request could not find host host.docker.internal. Please check the name and try again.
 PS C:\> ping gateway.docker.internal
 Ping request could not find host gateway.docker.internal. Please check the name and try again.
 PS C:\> ping google.com

 Pinging google.com [172.217.16.78] with 32 bytes of data:
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46

 Ping statistics for 172.217.16.78:
     Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
 Approximate round trip times in milli-seconds:
     Minimum = 31ms, Maximum = 31ms, Average = 31ms

C:\WINDOWS\system32>docker version
Client:
 Version:       18.03.0-ce
 API version:   1.37
 Go version:    go1.9.4
 Git commit:    0520e24
 Built: Wed Mar 21 23:06:28 2018
 OS/Arch:       windows/amd64
 Experimental:  false
 Orchestrator:  swarm

Server:
 Engine:
  Version:      18.03.0-ce
  API version:  1.37 (minimum version 1.24)
  Go version:   go1.9.4
  Git commit:   0520e24
  Built:        Wed Mar 21 23:21:06 2018
  OS/Arch:      windows/amd64
  Experimental: false

[noraab]

To provide more details: This happens on Docker Version 18.09.1 This does not happen on Docker Version 18.09.0

And some more findings:

• This does not happen if you use Docker's default network.
• This does happen, if you configure your own network within compose-file

I just defined my own network to have static IPs for each container and to pass this IP into the container as an environment - as I need it there for some RMI stuff configuration.

Now, I removed the custom network, and used the following code within my entrypoint to find out the dynamically assigned IP and write it into some variable:

setlocal enabledelayedexpansion
rem throw away everything except the IPv4 address line
for /f "usebackq tokens=*" %%a in (`ipconfig ^| findstr /i "ipv4"`) do (
  rem we have for example "IPv4 Address. . . . . . . . . . . : 192.168.42.78"
  rem split on : and get 2nd token
  for /f delims^=^:^ tokens^=2 %%b in ('echo %%a') do (
    rem we have " 192.168.42.78"
    set IP_ADDRESS=%%b
    set IP_ADDRESS=!IP_ADDRESS:~1!
    )
  )
)
(
endlocal
set IP_ADDRESS=%IP_ADDRESS%
)

Credits to this article on superuser

[MagicShoebox]

While playing around with something completely different, I found something possibly related.

The host names are injected into the container by this automatically executed docker command (taken from the Windows Event Log with docker daemon debug set to true): (192.168.XXX.XXX is the host IP)

debug: exec commandLine: cmd.exe /C "ECHO 192.168.XXX.XXX host.docker.internal >> %systemroot%\system32\drivers\etc\hosts & ECHO 192.168.XXX.XXX gateway.docker.internal >> %systemroot%\system32\drivers\etc\hosts" [...]

The POST call that triggered this command was this:

debug: form data: {"AttachStderr":true,"AttachStdin":false,"AttachStdout":true,"Cmd":["cmd.exe","/C","ECHO 192.168.XXX.XXX host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts \u0026 ECHO 192.168.XXX.XXX gateway.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts"],"Detach":false,"DetachKeys":"","Env":null,"Privileged":false,"Tty":false,"User":"Administrator","WorkingDir":""}

The key part is "User":"Administrator". On nanoserver:1803, the built-in Administrator account is enabled with no password and it works fine. On servercore:1803, the built-in Administrator account is disabled (with a password) and it rejects the command.

CreateProcess: failure in a Windows system call: The user name or password is incorrect. (0x52e) [Event Detail: Provider: 00000000-0000-0000-0000-000000000000] [Event Detail: Provider: 00000000-0000-0000-0000-000000000000] [Event Detail: onecore\vm\compute\management\orchestration\vmhostedcontainer\processmanagement.cpp(174)\vmcomputeagent.exe!00007FF7B648C00A: (caller: 00007FF7B645ECEA) Exception(2) tid(36c) 8007052E The user name or password is incorrect. CallContext:[\Bridge_ProcessMessage\ComputeSystemManager_ExecuteProcess\VmHostedContainer_ExecuteProcess] Provider: 00000000-0000-0000-0000-000000000000]

Steps to reproduce docker run --rm -it mcr.microsoft.com/windows/nanoserver:1803 (in container) type %systemroot%\system32\drivers\etc\hosts (Notice the hosts are listed correctly)

docker run --rm -it mcr.microsoft.com/windows/servercore:1803 (in container) type %systemroot%\system32\drivers\etc\hosts (Notice the hosts are missing)

Strangely, when I tried putting the "type ...hosts" directly after docker run, the hosts were missing from BOTH docker run --rm -it mcr.microsoft.com/windows/[flavor]:1803 cmd /C type %systemroot%\system32\drivers\etc\hosts (Notice the hosts are missing)

Unfortunately, this doesn't seem to actually fix the ping, merely the lookup.

Still, Docker should probably use "User":"ContainerAdministrator" instead of "User":"Administrator" when adding the hosts, at least on 1803. Unfortunately I can't test either flavor of 1809 to see if they behave the same way.

As a side note, if anyone wants to get the hosts working on servercore:

EDIT: In case it's not clear, this gives Administrator (aka root) a blank password, just like nanoserver. Be sure you understand the security implications.

# escape=`

FROM mcr.microsoft.com/windows/servercore:1803

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

RUN Enable-LocalUser Administrator; `
    SecEdit.exe /export /cfg secpol.cfg; `
    (Get-Content secpol.cfg).Replace('PasswordComplexity = 1', 'PasswordComplexity = 0') | `
    Out-File secpol.cfg; `
    SecEdit.exe /configure /db C:\Windows\Security\Local.sdb /cfg secpol.cfg /areas SECURITYPOLICY; `
    Remove-Item secpol.cfg; `
    Set-LocalUser Administrator -Password (New-Object SecureString)

[mx-bernhard]

RUN Enable-LocalUser Administrator; SecEdit.exe /export /cfg secpol.cfg; (Get-Content secpol.cfg).Replace('PasswordComplexity = 1', 'PasswordComplexity = 0') | Out-File secpol.cfg; SecEdit.exe /configure /db C:\Windows\Security\Local.sdb /cfg secpol.cfg /areas SECURITYPOLICY; Remove-Item secpol.cfg; Set-LocalUser Administrator -Password (New-Object SecureString)

We have an internal tool that is run within a container used for our CI builds based on 1803 that seems to be affected by the absence of an administrator account. Using the ps-script above we were able to get our container running again. But here is the peculiar thing:

• The problem (CreateProcess: failure in a Windows system call: The user name or password is incorrect. (0x52e)) occurs after a period of about two weeks after installing docker to the machine the first time or going back to factory defaults. Either of the two fixes the problem but once a certain period has passed the issue reappears
• When the issue reappears, using the powershell script above fixes the problem for a certain time period
• Once this has been executed on this machine in one container and by using "--rm", it works for all other containers. So we do not have to execute the powershell script each time we want a working container but apparently only once within the above mentioned period. So there seems to be some kind of escaping the sandbox and usage of a cache on the docker host here. We could also observe that without the powershell script, the hosts file still contains two entries when the container is working and no entries when it is not working.

EDIT:

• we suspect that the two weeks interval comes from a new dynamic ip adress of the docker host, this requires the hosts file to be altered as it contains the docker host ip adress
• changing the docker host file from inside a container seems to require an administrator account but only then
• updating the hosts file within the docker container seems to update the hosts file of the docker host and thus fixes the problem for a certain time period for all other containers as well

EDIT 2:

• observed with Docker Desktop 2.0.0.3 / Engine 18.09.2

[mx-bernhard]

Unfortunately, it was not just the dynamic ip address. We could not reproduce when our tool is causing the issue and when it is not.

[oatsoda]

It's a windows daemon limitation and needs to be fix there but we will add a workaround waiting MS to fix it upstream.

@ebriney Is this still the case? Can we track the MS issue anywhere? Thanks

[Kralizek]

Getting this issue

PS> docker run --rm -i microsoft/nanoserver:1709 ping host.docker.internal
Ping request could not find host host.docker.internal. Please check the name and try again.

Docker version

Client: Docker Engine - Community
 Version:           18.09.2
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        6247962
 Built:             Sun Feb 10 04:12:31 2019
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.2
  API version:      1.39 (minimum version 1.24)
  Go version:       go1.10.6
  Git commit:       6247962
  Built:            Sun Feb 10 04:28:48 2019
  OS/Arch:          windows/amd64
  Experimental:     true

[AndyVulhop]

I have the same issue with my container based on microsoft/aspnet:4.7.2-windowsservercore-1803. Trying to use host.docker.internal as the server for a db connection string to a sql database running locally on my host fails due to not being able to resolve host.docker.internal. I am also able to confirm that the host file inside the running container does not have records for host.docker.internal, nor for gateway.docker.internal. They are created and updated on my host machine's host file properly.

Please advise on a preferred workaround or a fix timetable.

PS C:\WINDOWS\system32> docker info
Containers: 13
 Running: 3
 Paused: 0
 Stopped: 10
Images: 15
Server Version: 18.09.2
Storage Driver: windowsfilter
 Windows:
Logging Driver: json-file
Plugins:
 Volume: local
 Network: ics l2bridge l2tunnel nat null overlay transparent
 Log: awslogs etwlogs fluentd gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 17134 (17134.1.amd64fre.rs4_release.180410-1804)
Operating System: Windows 10 Pro Version 1803 (OS Build 17134.677)
OSType: windows
Architecture: x86_64
CPUs: 8
Total Memory: 15.88GiB
Name: OH-DUB-LAP-48XS
ID: YEOQ:S5SV:YTJS:FJRW:CCE2:HZGV:SF2V:BXVR:AXZZ:P5AT:736C:KQZD
Docker Root Dir: C:\ProgramData\Docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: -1
 Goroutines: 53
 System Time: 2019-04-03T22:05:04.3714794-04:00
 EventsListeners: 1
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

[alexvaut]

I can confirm I can reproduce on 18.09.2 with a windows server core image, more precisely: image 4.7.2-20190409-windowsservercore-ltsc2019. Workaround script provided by @MagicShoebox fixes the issue.

[Cloudmersive]

Is this still not fixed? It's 2019 guys

[cpumanaz]

Why can the docker-compose/swarm style DNS server not be used here? The A record can be managed by the container host instead of a host file. All docker containers get this DNS server entry first with the host's DNS servers appended. Just set the A record to the primary IPv4 address of the host.

[Dweeberly]

Still broke for me too:

PS > docker run --rm -i microsoft/nanoserver:1709 ping host.docker.internal
Unable to find image 'microsoft/nanoserver:1709' locally
1709: Pulling from microsoft/nanoserver
407ada6e90de: Already exists
9ef95ce817ec: Pull complete
Digest: sha256:97272be4e3abcd71727095a6ab1948afc4c8412ad0649abfe60912651769a67d
Status: Downloaded newer image for microsoft/nanoserver:1709
Ping request could not find host host.docker.internal. Please check the name and try again.

PS > docker info
Containers: 6
 Running: 0
 Paused: 0
 Stopped: 6
Images: 6
Server Version: 18.09.2
Storage Driver: windowsfilter
 Windows:
Logging Driver: json-file
Plugins:
 Volume: local
 Network: ics l2bridge l2tunnel nat null overlay transparent
 Log: awslogs etwlogs fluentd gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 16299 (16299.637.amd64fre.rs3_release_svc.180808-1748)
Operating System: Windows 10 Enterprise Version 1709 (OS Build 16299.1029)
OSType: windows
Architecture: x86_64
CPUs: 8
Total Memory: 63.89GiB
Name: xxxxxxx
ID: QQQQ:37AP:xxxx:MLOQ:Z4FE:xxxx:yyyy:RRYJ:xxxx:ZCCR:KKKK:xxxx
Docker Root Dir: C:\ProgramData\Docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: -1
 Goroutines: 39
 System Time: 2019-05-13T16:54:40.6304645-04:00
 EventsListeners: 2
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

I'll note that it was working but I had an extra 20GB in windowsfilter that I couldn't get rid of so I did a "factory reset". After that a running container could not resolve DNS, I could tracert my host by ip, but not by name. I brought down another (unrelated) vm I had running in hyperv, reset docker desktop and found this command still failed:

PS C:\Users\grgran\docker\nanops> docker run --rm -i microsoft/nanoserver:1709 ping host.docker.internal
Ping request could not find host host.docker.internal. Please check the name and try again.

However, it did start resolving DNS and I could tracert www.google.com without issues. Don't know if this helps but it appears ping'ing host.docker.internal is not a reliable test.

[jorioux]

Pinging host.docker.internal is not a reliable test. A true DNS test would be to ping another docker-compose service.

The issue is present on 10.0.16299, and I confirm it works on 10.0.17763.

[kiiwii]

Pinging host.docker.internal is not a reliable test. A true DNS test would be to ping another docker-compose service. The issue is present on 10.0.16299, and I confirm it works on 10.0.17763.

agree: services of docker stack/swarm/compose, cannot ping to each other‘ internal DNS name from different swarm node. but also disagree: not working on 10.0.17763.253, confirem as above ↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

[danieldaeschle]

Is there any update? It's still not working.

[ghost]

I don't believe it will ever be. Try to compile the windows binary yourself, copy it over the other one and it should work. https://github.com/rgl/docker-ce-windows-binaries-vagrant

[Heurazio]

Any update on this? After it has been resolved for some time, the problem is back after the latest update to 2.1.0.1 (37199).

[Heurazio]

Reinstallation of Docker for Windows solved the problem.

[kmukul123]

I am still seeing the problem, I see people have commented about the hostfile, the hostfile has an entry on the docker machine but not on the container machine so will it work on the container? I have tried reinstalling restarting, resetting etc.

without this you cant have part of the solution working locally and part of it on docker containers.

[gillesgalipeau]

The problem still persist in our environment (moby 39110) since April. Does anyone have a fix for us?

[spex66]

Solution approach which seems to work is to write the gateway IP address (from within the container) to "drivers\etc\hosts" files, to make DNS resolving of "host.docker.internal" and "gateway.docker.internal" possible.

As the PS script from @kierzo has not worked for me using PowerShell Core

• https://github.com/docker/for-win/issues/1976#issuecomment-418151244 due to:

  Most of the modules that ship as part of Windows (for example, DnsClient, Hyper-V, NetTCPIP, Storage, etc.) and other Microsoft products including Azure and Office have not been explicitly ported to .NET Core yet. https://docs.microsoft.com/en-us/powershell/scripting/whats-new/what-s-new-in-powershell-core-60?view=powershell-6

here is a minimum version working for PWSH Core v6 (for me):

$hostsFile = "${env:windir}\system32\drivers\etc\hosts"
$ip = (ipconfig | where-object {$_ –match "Default Gateway"} | foreach-object{$_.Split(":")[1]})
$addtohost = "`n#Added by Docker for Windows`n$ip    host.docker.internal`n$ip    gateway.docker.internal`n# End of section`n"
echo $addtohost | Out-File -Append -filepath $hostsFile

added it to our entrypoint.ps1 script and from there one used host.docker.internal to connect to services running on host (outside container).

Now back to work after spending an hour to dig through all posts and options around this issue :)

Waiting for a proper solution from MSFT to replace this hack.

[ki11roy]

I am experiencing a random docker build failures (~40 minutes long builds on a build machine) and it seems that those somehow connected with this error (whole build fails because of the failed ECHO command ?)

[23:03:06.612][WindowsDaemon     ][Info   ] debug: start() completed [module=libcontainerd namespace=moby container=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd
04ef0c2d3e32039e3823ba0]
[23:03:06.612][WindowsDaemon     ][Info   ] sending event [module=libcontainerd namespace=moby event-info={eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3
e32039e3823ba0 init 25660 0 0001-01-01 00:00:00 +0000 UTC false <nil>} container=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0 event=start
]
[23:03:06.612][WindowsDaemon     ][Info   ] debug: Calling HEAD /_ping
[23:03:06.612][WindowsDaemon     ][Info   ] debug: Calling GET /v1.40/containers/eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0/json
[23:03:06.612][WindowsDaemon     ][Info   ] debug: Calling POST /v1.40/containers/eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0/exec
[23:03:06.612][WindowsDaemon     ][Info   ] debug: form data: {"AttachStderr":true,"AttachStdin":false,"AttachStdout":true,"Cmd":["cmd.exe","/C","ECHO 10.128
.2.243    host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts \u0026 ECHO 10.128.2.243    gateway.docker.internal \u003e\u003e %sys
temroot%\\system32\\drivers\\etc\\hosts"],"Detach":false,"DetachKeys":"","Env":null,"Privileged":false,"Tty":false,"User":"Administrator","WorkingDir":""}
[23:03:06.612][WindowsDaemon     ][Info   ] debug: Calling POST /v1.40/exec/46dde09293222d87ac30f356167f1074138b34f064a8b8e9f6db44e85a4388b0/start
[23:03:06.612][WindowsDaemon     ][Info   ] debug: form data: {"Detach":false,"Tty":false}
[23:03:06.612][WindowsDaemon     ][Info   ] debug: starting exec command 46dde09293222d87ac30f356167f1074138b34f064a8b8e9f6db44e85a4388b0 in container eb719a
6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0
[23:03:06.612][WindowsDaemon     ][Info   ] debug: exec commandLine: cmd.exe /C "ECHO 10.128.2.243    host.docker.internal >> %systemroot%\system32\drivers\e
tc\hosts & ECHO 10.128.2.243    gateway.docker.internal >> %systemroot%\system32\drivers\etc\hosts" [exec=46dde09293222d87ac30f356167f1074138b34f064a8b8e9f6d
b44e85a4388b0 module=libcontainerd namespace=moby container=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0]
[23:03:06.612][WindowsDaemon     ][Info   ] debug: hcsshim::ComputeSystem::CreateProcess - Begin Operation [cid=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef
0c2d3e32039e3823ba0]
[23:03:06.612][WindowsDaemon     ][Info   ] debug: HCS ComputeSystem Process Document [cid=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0 j
son={"CommandLine":"cmd.exe /C \"ECHO 10.128.2.243    host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts \u0026 ECHO 10.128.2.243
   gateway.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts\"","User":"Administrator","WorkingDirectory":"C:\\build","Environment":{"
COMPLUS_NGenProtectedProcess_FeatureEnabled":"0","GIT_ASKPASS":"C:\\git-credential-helper.bat","HOST_ADDRESS":"10.128.2.243","HOST_TOKEN":"683cb9d70552cbf762
20861aa96ba28f","NUGET_VERSION":"4.4.1","ROSLYN_COMPILER_LOCATION":"C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\MSBuild\\Current\\Bin
\\Roslyn"},"CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0]}]
[23:03:06.612][WindowsDaemon     ][Info   ] debug: attach: stderr: begin
[23:03:06.612][WindowsDaemon     ][Info   ] debug: attach: stdout: begin
[23:03:06.612][WindowsDaemon     ][Info   ] debug: HCS Result [json={"Error":-2147023570,"ErrorMessage":"The user name or password is incorrect."}]
[23:03:06.612][WindowsDaemon     ][Error  ] hcsshim::ComputeSystem::CreateProcess - End Operation - Error [cid=eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0
c2d3e32039e3823ba0 error=CreateProcess eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0: The user name or password is incorrect.

and after that:

[23:03:06.613][WindowsDaemon     ][Error  ] Error running exec 46dde09293222d87ac30f356167f1074138b34f064a8b8e9f6db44e85a4388b0 in container: container eb719
a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0 encountered an error during CreateProcess: failure in a Windows system call: The user name or pas
sword is incorrect. (0x52e) extra info: {"CommandLine":"cmd.exe /C \"ECHO 10.128.2.243    host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\
etc\\hosts \u0026 ECHO 10.128.2.243    gateway.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts\"","User":"Administrator","WorkingDir
ectory":"C:\\build","Environment":{"COMPLUS_NGenProtectedProcess_FeatureEnabled":"0","GIT_ASKPASS":"C:\\git-credential-helper.bat","HOST_ADDRESS":"10.128.2.2
43","HOST_TOKEN":"683cb9d70552cbf76220861aa96ba28f","NUGET_VERSION":"4.4.1","ROSLYN_COMPILER_LOCATION":"C:\\Program Files (x86)\\Microsoft Visual Studio\\201
9\\BuildTools\\MSBuild\\Current\\Bin\\Roslyn"},"CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0]}
[23:03:06.613][WindowsDaemon     ][Info   ] debug: attach: stdout: end
[23:03:06.613][WindowsDaemon     ][Info   ] debug: attach: stderr: end
[23:03:06.613][WindowsDaemon     ][Info   ] debug: attach done
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Calling GET /v1.40/exec/46dde09293222d87ac30f356167f1074138b34f064a8b8e9f6db44e85a4388b0/json
[23:03:06.613][WindowsDaemon     ][Info   ] debug: clean 1 unused exec commands
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Client context cancelled, stop sending events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Calling GET /v1.40/events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: clean 3 unused exec commands
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Client context cancelled, stop sending events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Calling GET /v1.40/events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Client context cancelled, stop sending events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Calling GET /v1.40/events
[23:03:06.613][WindowsDaemon     ][Info   ] debug: Build cancelled, killing and removing container: eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e
3823ba0
[23:03:06.614][WindowsDaemon     ][Info   ] debug: Sending kill signal 9 to container eb719a6ee48e274ae06cc35bff74e1d8e9d4c93dd04ef0c2d3e32039e3823ba0

and the result is:

[23:03:07.641][WindowsDaemon     ][Info   ] debug: hcsshim::DestroyLayer - succeeded [path=D:\docker_windows\windowsfilter\0c92737e7b38f61d3bb4aed66d69280aa7
89283a35da07f5ce7802784ca8f6a6-removing]
[23:03:07.641][WindowsDaemon     ][Info   ] debug: Build cancelled (build cancelled): a non-zero code from ContainerWait: 3221225473

I've tried to apply @MagicShoebox script, but with no luck.

> docker info
Client:
 Debug Mode: false
 Plugins:
  app: Docker Application (Docker Inc., v0.8.0)
  buildx: Build with BuildKit (Docker Inc., v0.3.0-5-g5b97415-tp-docker)

Server:
 Containers: 3
  Running: 0
  Paused: 0
  Stopped: 3
 Images: 115
 Server Version: 19.03.2
 Storage Driver: windowsfilter (windows) lcow (linux)
  Windows:
  LCOW:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics l2bridge l2tunnel nat null overlay transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: inactive
 Default Isolation: process
 Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
 Operating System: Windows 10 Enterprise Version 1809 (OS Build 17763.737)
 OSType: windows
 Architecture: x86_64
 CPUs: 12
 Total Memory: 63.95GiB
 Name: BY1-CABuilds-01
 ID: RS5M:6DEO:RNHL:MYF4:CBRW:T6XD:E6CT:HMVF:HRXN:FSKU:MSJN:5GRM
 Docker Root Dir: D:\docker_windows
 Debug Mode: true
  File Descriptors: -1
  Goroutines: 33
  System Time: 2019-09-29T23:25:41.1348512+03:00
  EventsListeners: 1
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

[brianharwell]

I am updating the hosts file when the container starts but there is an issue with that because the ip resolved in a container where hosts.docker.internal works is different from the gateway ip address used in the workaround.

[brianharwell]

I think I found the cause of the issue, when looking at the Docker logs for a different issue I found this...

[Event Detail: onecore\vm\compute\management\orchestration\vmhostedcontainer\processmanagement.cpp(173)\vmcomputeagent.exe!00007FF6000D9ADB: (caller: 00007FF60008DF1A) Exception(2) tid(4c4) 8007052E The user name or password is incorrect.
    CallContext:[\Bridge_ProcessMessage\VmHostedContainer_ExecuteProcess] 
 Provider: 00000000-0000-0000-0000-000000000000]
(extra info: {"CommandLine":"cmd.exe /C \"ECHO 192.168.1.204    host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts \u0026 ECHO 192.168.1.204    gateway.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts\"","User":"Administrator","WorkingDirectory":"/","Environment":{"COMPLUS_NGenProtectedProcess_FeatureEnabled":"0","ROSLYN_COMPILER_LOCATION":"c:\\\\RoslynCompilers\\\\tools"},"CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0]})]

The important thing to note is the error The user name or password is incorrect and the command seems to be trying to add host.docker.internal into the hosts file.

[f2calv]

I have been successfully building windows server images without issue on Windows 10 Enterprise Insiders edition for the past month or two, last night I upgraded to the latest Win10 insiders edition and this DNS resolving issue has begun to manifest itself. I've submitted an insiders bug report.

Note: copying in a fake hosts file fixed the issue, however that isn't a viable solution as most of you have already found.

[docker-robott]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

[Kralizek]

what is the status of this issue?

[ghost]

They probably don't want to fix it

[brunnotelma]

/remove-lifecycle stale

[andrejsobkowski]

For what it's worth, it's not working in my environment either:

• Windows 10 Pro
• Docker version 2.1.0.5 (40693)
• Docker Engine 19.03.5

Executing docker exec -it CONTAINERID ping docker.host.internal returns Ping request could not find host docker.host.internal. Please check the name and try again.

Same happens with "docker.gateway.internal".

Executing docker exec -it CONTAINERID ping www.google.com works fine.

I agree with previous comments stating that adding an entry to the hosts file may not be appropriate; in our case we may lose connectivity and reconnect later with a different IP. Though I'm not much of a network expert.

[llinke]

Executing docker exec -it CONTAINERID ping docker.host.internal returns Ping request could not find host docker.host.internal. Please check the name and try again.

Same happens with "docker.gateway.internal".

I guess it's host.docker.internal, not docker.host.internal...

[andrejsobkowski]

I guess it's host.docker.internal, not docker.host.internal...

@llinke I guess you're so very right. Thank you for taking the time to answer (and apologies for me not being able to read words in the right order).

[brunnotelma]

I have some Windows containers that were resolving host.docker.internal and gateway.docker.internal, along with some other Windows containers that weren't resolving those names, all in the same docker-compose stack.

After the last update, none of them are working anymore for some reason, so the workaround I did (based on some comments in this same issue) is this:

Dockerfile:

ENTRYPOINT ["powershell", "/docker-entrypoint.ps1;"]

Entrypoint script:

# DNS Entries
Write-Host("Configuring DNS entries...")
C:\fix-hosts.ps1

fix-hosts.ps1:

$hostsFile = "C:\Windows\System32\drivers\etc\hosts"

try {
    $DnsEntries = @("host.docker.internal", "gateway.docker.internal")
    # Tries resolving names for Docker
    foreach ($Entry in $DnsEntries) {
        # If any of the names are not resolved, throws an exception
        Resolve-DnsName -Name $Entry -ErrorAction Stop
    }

    # If it passes, means that DNS is already configured
    Write-Host("DNS settings are already configured.")
} catch {
    # Gets the gateway IP address, that is the Host's IP address in the Docker network
    $ip = (ipconfig | where-object {$_ -match "Default Gateway"} | foreach-object{$_.Split(":")[1]}).Trim()
    # Read the current content from Hosts file
    $src = [System.IO.File]::ReadAllLines($hostsFile)
    # Add the a new line after the content
    $lines = $src += ""

    # Check the hosts file and write it in if its not there...
    if((cat $hostsFile | Select-String -Pattern "host.docker.internal") -And (cat $hostsFile | Select-String -Pattern "gateway.docker.internal")) {
        For ($i=0; $i -le $lines.length; $i++) {
            if ($lines[$i].Contains("host.docker.internal"))
            {
                $lines[$i] = ("{0} host.docker.internal" -f $ip)
                $lines[$i+1] = ("{0} gateway.docker.internal" -f $ip)
                break
            }
        }
    } else {
        $lines = $lines += "# Added by Docker for Windows"
        $lines = $lines += ("{0} host.docker.internal" -f $ip)
        $lines = $lines += ("{0} gateway.docker.internal" -f $ip)
        $lines = $lines += "# End of section"
    }
    # Writes the new content to the Hosts file
    [System.IO.File]::WriteAllLines($hostsFile, [string[]]$lines)

    Write-Host("New DNS settings written successfully.")
}

When the container starts, before executing the main command it starts the fix-hosts.ps1 script.

What the script does is: 1) Checks if the names can be resolved.

• If they can, assumes it's configured properly. 2) If not, looks up in the hosts file if there are entries there already.
• If there is, replace them with the gateway IP address, which is the host's IP address. 3) If not, just adds the new entries for both gateway.docker.internal and host.docker.internal in the hosts file. 4) Write the new content to the hosts file. 5) The end.

That's it! It's the neatest approach I could get so far without having to hack it too much, and it works with pretty much any Windows image.

[garrachachraf]

Are you using AzureAD by any chance? i had the same problem and it was a permission problem, i fixed it with this script : $path = "C:\ProgramData\docker" $permission = "AzureAD\you@yourcompany.com","FullControl","Allow" (Get-Acl $path).SetAccessRule((New-Object System.Security.AccessControl.FileSystemAccessRule $permission)) | Set-Acl $path

[karlholbrook]

I am seeing the same results as @brianharwell. I can see "The user name or password is incorrect." in the docker daemon logs. This is happening when the attempt is made to add 'host.docker.internal' to the container hosts file. Note the User is Administrator.

This only fails with servercore (FROM mcr.microsoft.com/windows/servercore).

If I use exactly the same docker file but change to nanoserver (FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-nanoserver-1809) it works perfectly. I can see in the docker daemon log the hosts file being successfully updated with host.docker.internal.

The User is showing as Administrator in both log files.

I can only conclude that docker is trying to pass the same password to the servercore and nanoserver images but the passwords are in fact not the same on both images?

Below is the extract from the failed servercore daemon log startup,

[08:36:16.473][WindowsDaemon ][Info ] debug: HCS ComputeSystem Process Document [cid=17d0c4096b6e94ba507f867283e17d54aa4270643cc29945904ba1f9c6bde0d1 json={"CommandLine":"cmd.exe /C \"ECHO 10.3.112.207 host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts \u0026 ECHO 10.3.112.207 gateway.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts\"","User":"Administrator","WorkingDirectory":"C:\\app","Environment":{"ASPNETCORE_URLS":"http://+:80","DOTNET_RUNNING_IN_CONTAINER":"true","DOTNET_SDK_VERSION":"3.1.100","DOTNET_USE_POLLING_FILE_WATCHER":"true","NUGET_XMLDOC_MODE":"skip"},"CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0]}] [08:36:16.527][WindowsDaemon ][Info ] debug: HCS Result [json={"Error":-2147023570,"ErrorMessage":"The user name or password is incorrect.","ErrorEvents":[{"Message":"","Provider":"00000000-0000-0000-0000-000000000000","EventId":0},{"Message":"","Provider":"00000000-0000-0000-0000-000000000000","EventId":0},{"Message":"onecore\\vm\\compute\\management\\orchestration\\vmhostedcontainer\\processmanagement.cpp(173)\\vmcomputeagent.exe!00007FF64CB29D2B: (caller: 00007FF64CADE13A) Exception(2) tid(4dc) 8007052E The user name or password is incorrect.\r\n CallContext:[\\Bridge_ProcessMessage\\VmHostedContainer_ExecuteProcess] \n","Provider":"00000000-0000-0000-0000-000000000000","EventId":0}]}]

[DoxaLogosGit]

I've having the same problem with FROM mcr.microsoft.com/windows/servercore:1803. It works with nanoserver:1803, but I need powershell for my images and nanoserver:1803 doesn't have powershell. If I try to pull mcr.microsoft.com/powershell, the image isn't compatible with my version of Windows 10 Pro. So, I'm kind of stuck. If @karlholbrook is correct, this seems like it would be an easy fix on MS's side to correct the password issue in servercore:1803

[bklebe]

Have some ASP apps I would like to containerize which rely on SQL Server. This is a blocker for local development, do we know where the issue lies?

[STotev]

I've had issues with host.docker.internal created accidentally by me. The issue was that I needed to fix another issue related to nodejs stucking while installing dependencies. The issue was resolved by adding a second DNS to already available 8.8.8.8. The dependencies installation started working but apparently that caused this issue with host.docker.internal.

So I traced back what I've done recently and remove the additional DNS and left only 8.8.8.8. Restarted Docker and now works fine.

Although, there will be an issue again with nodejs, so this somehow needs to be resolved.

My Docker info is: Version: 2.3.0.3 Engine: 19.03.8

[mbrankintrintech]

Still seeing this behaviour. Windows 10, ServerCore.

[docker-robott]

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

[tihomir-kit]

/remove-lifecycle stale

[docker-robott]

Issues go stale after 90 days of inactivity. Mark the issue as fresh with /remove-lifecycle stale comment. Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows. /lifecycle stale

[tihomir-kit]

/remove-lifecycle stale

[keystroke]

Still getting this issue, my containers (same images) randomly fail on startup. sometimes they start, sometimes they hang in "created" state and application log filled with repeated errors like below. I am running windows containers on windows 10 surface book laptop, calling docker run from PowerShell ISE elevated (running as admin) and still hit this issue at least 50% of the time, without changing anything in my images:

exec's CreateProcess() failed [module=libcontainerd namespace=moby container=a06ed0f5c4f8b94c5462d4d64c5b370e4db7cc95422dda325f1fc44abf186a6c exec=a6ccdf64ab7e3b48360dc82630cbb028263e1232ace044f1285d2b71e3a489da error=container a06ed0f5c4f8b94c5462d4d64c5b370e4db7cc95422dda325f1fc44abf186a6c encountered an error during hcsshim::System::CreateProcess: failure in a Windows system call: The user name or password is incorrect. (0x52e)
[Event Detail:  Provider: 00000000-0000-0000-0000-000000000000]
[Event Detail:  Provider: 00000000-0000-0000-0000-000000000000]
[Event Detail: onecore\vm\compute\management\orchestration\vmhostedcontainer\processmanagement.cpp(173)\vmcomputeagent.exe!00007FF7E2119D2B: (caller: 00007FF7E20CE13A) Exception(2) tid(4d4) 8007052E The user name or password is incorrect.
    CallContext:[\Bridge_ProcessMessage\VmHostedContainer_ExecuteProcess] 
 Provider: 00000000-0000-0000-0000-000000000000] extra info: {"CommandLine":"cmd.exe /C \"ECHO 192.168.0.117    host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts \u0026 ECHO 192.168.0.117    gateway.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts\"","User":"Administrator","WorkingDirectory":"/","CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0]}]

[brianharwell]

Is anyone looking into the root cause of this?

[BogusDude99]

/remove-lifecycle stale

[gabrielpagu]

Docker 3.3.3 Same issue

[DeoluA]

Updated to Docker Desktop 3.3.3 a few days ago, and this issue seems to have crop up again (hostname gateway.docker.internal was resolving fine before then). Perhaps something broke in this latest update?

[netpoetica]

I am on Docker 3.3.3 and I also am seeing this issue - anyone know a specific version of Docker where it actually did work?

[KyronSr]

I am on "Docker version 20.10.6, build 370c289" and have the same problem. I've used @brunnotelma 's workaround successfully, thanks for that!

[netpoetica]

If I use https://docs.docker.com/docker-for-windows/release-notes/#docker-desktop-community-2501 with docker.for.win.localhost or host.docker.internal I see the same issues - @brunnotelma's workaround did not fix this issue with Docker 3.x either. I am tracking this issue with reproducible images here: https://github.com/envoyproxy/envoy/issues/16710 in case it is of interest to the Docker team