hcsshim: timeout waiting for notification extra info

[pradley]

When trying to run my Rabbit image from my compose file I get this error below.

ERROR: for bin_Rabbit_1 Cannot start service Rabbit: container 4c49c5ce1c9be3f3deca474403b7a9df44ac09151bae5126c60768cf01767428 encountered an error during CreateContainer: hcsshim: timeout waiting for notification extra info: {"SystemType":"Container","Name":"4c49c5ce1c9be3f3deca474403b7a9df44ac09151bae5126c60768cf01767428","Owner":"docker","IgnoreFlushesDuringBoot":true,"LayerFolderPath":"C:\ProgramData\Docker\windowsfilter\4c49c5ce1c9be3f3deca474403b7a9df44ac09151bae5126c60768cf01767428","Layers":[{"ID":"dee86350-459f-580c-ae1e-fcc1bee0baa2","Path":"C:\ProgramData\Docker\windowsfilter\c056770fbd90992091b4d16fe9c2b608d739689b9b8d6f8b24edc6ecd36cfb3c"},{"ID":"cf555c82-e82c-5103-9328-02e79a453583","Path":"C:\ProgramData\Docker\windowsfilter\a1a2460d2ca841aac7e92a84fba5af4fa61274d960eff41ac1d5384bfff30efd"},{"ID":"d21d7c62-9717-5f8c-b40b-2e20f2c99b04","Path":"C:\ProgramData\Docker\windowsfilter\80fbafed497f150bbbdec469621cdb01494ac302443ee66be797ec167a88607c"},{"ID":"34db3c60-39de-51dc-bfbf-e9c907c6e86b","Path":"C:\ProgramData\Docker\windowsfilter\a10b48318221d634f2df7d4f6bbd8c4c24170cf92c8b54eaf15d21c6d12efe45"},{"ID":"5fb20f07-e3dc-5034-86a3-f1103c3377c3","Path":"C:\ProgramData\Docker\windowsfilter\4cfafd1cab11aa92b130ef6c3ed9a0f41d89ae500255424901c50408edbeb45b"},{"ID":"dbe867e1-5401-5747-9edc-02780de37593","Path":"C:\ProgramData\Docker\windowsfilter\802c32841d34c97eb63462666065b9fec57a4ea2aa9373bffe1425790078f4d1"},{"ID":"12b89f02-8d16-598e-86b0-f7c17f82612e","Path":"C:\ProgramData\Docker\windowsfilter\53801ceea5ae8088b78b6af799956087740807560aece1f504f1bb3c40efdee6"},{"ID":"28fb85e9-113c-5072-9ca5-d7de54103a5c","Path":"C:\ProgramData\Docker\windowsfilter\f6b75b2ad9713292ba588c5cd81a1efa8aadbebd6f23811cdd13327f0504d1fe"},{"ID":"2dea95ad-c46c-5ab3-a9eb-cac8df2c1451","Path":"C:\ProgramData\Docker\windowsfilter\33dcdab745abced6c32b832722b708284da1cc5ab049fe418af8c5ae42659670"},{"ID":"0df81b3f-652b-596c-8223-acaab6087dad","Path":"C:\ProgramData\Docker\windowsfilter\2c2d427c6268e0520729be4107b6c3839fd63ffdd05236a5cc9cbdc6b3ce7190"},{"ID":"de2d5623-089a-5d2c-944e-9246b670b4e6","Path":"C:\ProgramData\Docker\windowsfilter\16f6cc2dd45bfbd1be8b3255612f7740731744ee6f6dbb3f049eefd535df962f"},{"ID":"fd01999c-0b74-515c-ad9b-71b4236015eb","Path":"C:\ProgramData\Docker\windowsfilter\8b8f0948e6aa5ad08b3042a03c0bbd5ffd971d9f7e26d052d5afde4abb1837ad"},{"ID":"fc9ade98-724f-5fbb-8363-1ba433028c3d","Path":"C:\ProgramData\Docker\windowsfilter\abe09c74ed9ef55cde9a138b9b1cba2a3d987a43d2a3d492018a9e8b2d2bd94e"},{"ID":"60162656-a118-5f4a-a081-7114cce85437","Path":"C:\ProgramData\Docker\windowsfilter\1d6314999ada0560529b2fbbb14d4f35341cd2911959c0fe9be85d736ff3ca29"},{"ID":"fa0bbe42-6d85-531a-bfcb-1822906ff2c3","Path":"C:\ProgramData\Docker\windowsfilter\beb26da51fdda5d9d72ba60069d9b65fe35052013fac5f765775fc6e9224bf6b"},{"ID":"1d9b3c2c-68e4-5e56-82a9-3073faa6b72a","Path":"C:\ProgramData\Docker\windowsfilter\f420d7b6053c27051b688473386b8b621cf2a6f3ecca9f2600dfda0f2de20a92"}],"MemoryMaximumInMB":3072,"HostName":"4c49c5ce1c9b","HvPartition":true,"EndpointList":["221d7a6a-4b70-4f8f-bccd-afa1b6deb906"],"HvRuntime":{"ImagePath":"C:\ProgramData\Docker\windowsfilter\beb26da51fdda5d9d72ba60069d9b65fe35052013fac5f765775fc6e9224bf6b\UtilityVM"},"AllowUnqualifiedDNSQuery":true}

If I run the rabbit image using the docker run command the image will work fine. Only through compose it gives this error! I understand that its timing out waiting for a notification. I just dont know how exactly to go about fixing this or what to look at.

[darstahl]

This timeout is caused by the HcsCreateComputeSystem (Windows API) call being done async and hanging for some reason. Since the hang is in the platform, not Go, we'll need to collect an HCS trace to debug this.

EDIT: See below for updated trace instructions

[dermeister0]

I have the same problem.

FROM microsoft/windowsservercore:1709

RUN mkdir C:\Build

PS C:\Users\anton\Desktop\DockerTest> docker build .
Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM microsoft/windowsservercore:1709
 ---> be1324f21832
Step 2/2 : RUN mkdir C:\Build
 ---> Running in b568056319be
container b568056319bef54ed28031228d38362153da8775b715b7a3ab11af2e772b5da6 encountered an error during CreateContainer: hcsshim: timeout waiting for notification extra info: {"SystemType":"Container","Name":"b568056319bef54ed28031228d38362153da8775b715b7a3ab11af2e772b5da6","Owner":"docker","IgnoreFlushesDuringBoot":true,"LayerFolderPath":"D:\\Docker2\\windowsfilter\\b568056319bef54ed28031228d38362153da8775b715b7a3ab11af2e772b5da6","Layers":[{"ID":"8a1cd827-e3f3-52e5-a2a2-18edd87a32c3","Path":"D:\\Docker2\\windowsfilter\\a4c1853a70a019beeb62b7184361f855be533ce1c933dd49158420bc05b17fa2"},{"ID":"0509ff50-38ef-5064-9ef8-93099eb63817","Path":"D:\\Docker2\\windowsfilter\\d532172bb554c431d4f4b6dfca2474e96ec1f563f0b3b26c30d210f0ef080a36"}],"HostName":"b568056319be","HvPartition":true,"EndpointList":["707ba3e2-0e0f-479f-845b-137a4c0efac1"],"HvRuntime":{"ImagePath":"D:\\Docker2\\windowsfilter\\a4c1853a70a019beeb62b7184361f855be533ce1c933dd49158420bc05b17fa2\\UtilityVM"},"AllowUnqualifiedDNSQuery":true}
PS C:\Users\anton\Desktop\DockerTest>

@darrenstahlmsft I sent a trace to you.

[darstahl]

Got it. Forwarding to the internal team to take a look.

[mvsouza]

Hi @darrenstahlmsft! Do you have any updates on the issue? I've downloaded the WPA to analyze my log. I'm trying to make any sense of it.

[darstahl]

No updates yet. So far this only seems to be affecting Hyper-V isolation containers on RS3 (1709) Windows builds. If anyone is seeing this on other hosts or Windows Server containers, let me know. I'll keep this thread updated.

[GrimSmiler]

Hey! After a recent Windows update, our docker hosting machine (Win 10) has updated, I am also getting this error.

Are there any updates?

[kavyarani7]

Hey @darrenstahlmsft ,

I am running docker on Windows 10 professional. I also have the similar issue when i am trying to install my node dependencies using "npm install".

Hope you find the cause of this issue soon.

[ciaranodonnell]

I'm having the same issue on Win10 Pro with Windows server images. I am also unable to run Linux images because: Docker hv-sock proxy (vsudd) is not reachable at Docker.Backend.ContainerEngine.Linux.ConnectToVsud(TaskCompletionSource`1 vmId) in C:\gopath\src\github.com\docker\pinata\win\src\Docker.Backend\ContainerEngine\Linux.cs:line 293 at Docker.Backend.ContainerEngine.Linux.DoStart(Settings settings, String daemonOptions) in C:\gopath\src\github.com\docker\pinata\win\src\Docker.Backend\ContainerEngine\Linux.cs:line 260 at Docker.Backend.ContainerEngine.Linux.Start(Settings settings, String daemonOptions) in C:\gopath\src\github.com\docker\pinata\win\src\Docker.Backend\ContainerEngine\Linux.cs:line 130 at Docker.Core.Pipe.NamedPipeServer.<>c__DisplayClass9_0.b__0(Object[] parameters) in C:\gopath\src\github.com\docker\pinata\win\src\Docker.Core\pipe\NamedPipeServer.cs:line 47 at Docker.Core.Pipe.NamedPipeServer.RunAction(String action, Object[] parameters) in C:\gopath\src\github.com\docker\pinata\win\src\Docker.Core\pipe\NamedPipeServer.cs:line 145

not sure if that's related. However Docker on windows is now unusable

[GrimSmiler]

Guys, small update.

For me this happens if I execute the following sequence:

docker-compose -f docker-compose.yml -f docker-compose.staging.yml -p MyServer down --rmi local 
docker-compose -f docker-compose.yml -f docker-compose.staging.yml -p MyServer build --no-cache
docker-compose -f docker-compose.yml -f docker-compose.staging.yml -p MyServer up -d --force-recreate

if i skip the build --no-cache, docker builds and starts up all fine.

maybe this will be of help. This is not a 100% fix, but might work every nth time.

still would like to know the actual problem and the fix :)

[NReilingh]

I'm not using docker compose, just trying to run windows containers on Windows 10. I was getting this error earlier, I think when I was trying to allocate too much memory. But last night Windows installed some updates and now I can't run my container at all. Just trying to do the basic docker run for microsoft/mssql-server-windows-developer.

I'm pretty sure my system was fully up-to-date prior to yesterday's updates, and the only things that changed were the Tuesday security patches, but I'm not sure I can verify that yet. This is all in a VM and I do have a backup from a few days ago, so I might be able to try that.

Anything else I could do to provide diagnostic information?

[jorisdg]

Any updates on this issue or how to work around it?

[darstahl]

We're tracking a few internal issues that all result in this same error message. In order to better understand the different causes, I'm uploading a new trace profile (HcsTraceProfile2.wprp) that will trace more details than the above version.

Wpr.exe -start HcsTraceProfile2.wprp!Hcs -filemode

<repro the issue here>

Wpr.exe -stop HcsTraceTimeout.etl "Some description"

Also get the event logs with the following:

wevtutil.exe epl Microsoft-Windows-Hyper-V-Worker-Analytic worker-analytic.evtx
wevtutil.exe epl Microsoft-Windows-Hyper-V-Worker-Admin worker-admin.evtx

Note that the resulting HcsTraceTimeout.etl may contain personal information, such as path names accessed on the host, current running executables, etc, so I suggest sending it to me directly at darst@microsoft.com.

If you can send all the above files, then we can better make sure that all the underlying issues are understood.

[tomashalac]

Any updates on how to work around it?

[tomashalac]

I moved everything to a new visual studio solution and it worked.

[aroxby]

Any updates? I just installed docker the other day and I'm having this problem with all windows containers. Linux containers work fine. Are you still looking for those files? I should have the time to generate them this week.

[darstahl]

There are multiple issues that result in this same error message. I'd still be happy to take the above files so I can make sure we're addressing all the underlying issues that might cause this.

[aroxby]

I tried this again today and did not experience the issue. Maybe the computer just needed a restart?

[dermeister0]

@darrenstahlmsft

I'm uploading a new trace profile (HcsTraceProfile2.wprp)

I sent an updated trace to you.

[ClintSulis]

@darrenstahlmsft

I sent a trace to you as well. Hope it helps!

[darstahl]

@dermeister0 @ClintSulis I had a chance to look at both your traces today, they are both the same issue, and we understand the problem. I don't have any timelines or workarounds at this time, but know we are working to get our fix out via Windows Servicing.

[ClintSulis]

Thanks! You are an awesome dude.

---

From: Darren Stahl notifications@github.com Sent: Friday, March 30, 2018 3:57:10 PM To: Microsoft/hcsshim Cc: ClintSulis; Mention Subject: Re: [Microsoft/hcsshim] hcsshim: timeout waiting for notification extra info (#152)

@dermeister0https://github.com/dermeister0 @ClintSulishttps://github.com/ClintSulis I had a chance to look at both your traces today, they are both the same issue, and we understand the problem. I don't have any timelines or workarounds at this time, but know we are working to get our fix out via Windows Servicing.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/Microsoft/hcsshim/issues/152#issuecomment-377632527, or mute the threadhttps://github.com/notifications/unsubscribe-auth/Aj2yMCcLvFNXgHt9bYWPjrvxsE2vKrLuks5tjqo2gaJpZM4RJEoL.

[RyanJLind64]

Does anybody have a fix for this? It has been blocking me for 3 days now. I have been de-installing windows updates to try to get back to a state where I don't encounter this problem.

[rickbutton]

Any update on this? Just ran into this exact error today on a developer's machine

[daghb]

Same error here. Win 10 latest docker host with several windows server core images

[RyanJLind64]

Please, some kind of update would be greatly appreciated.

[adamrehn]

In my testing under Win10, it seems that altering the memory limit for containers between subsequent Docker runs (e.g using -m 6GB for one run and then -m 8GB for the next) can reduce the frequency with which this error occurs. Since Win10 always uses Hyper-V isolation, I'm guessing that changing the memory limit forces Docker to provision a new Hyper-V VM, preventing it from re-using an existing VM that may have fallen victim to whatever condition triggers the timeout.

[Kuinox]

Hi, I alsot get this error, but mostly when the system is out of ram and is swapping.

[mback2k]

I am running into the same issue from time to time by building the Dockerfile hosted here: https://github.com/mback2k/docker-windows-buildbot-msys2/blob/master/Dockerfile

[ssell]

Can confirm that this error still occurs as of the Windows 10 April 2018 update (Version 1803) with Docker CE Edge Version 18.05.0-ce-rc1-win63 (17439).

Though when I encounter the error, it is slightly different from what others have mentioned in this issue. I receive it during image creation with Docker timing out while waiting for the temporary container to shutdown.

The command 'cmd /S /C powershell c:/content/install.ps1' returned a non-zero code: 4294967295: failed to shutdown container: container 429047b1fdd4adcd8099081ecf0a2cee26fcb819121f2c90710b4fe2a2a58b63 encountered an error during WaitTimeout: hcsshim: timeout waiting for notification

Where install.ps1 simply runs a WiX-based installer bundle:

Start-Process -FilePath "c:/content/installer.exe" -ArgumentList "-q" -PassThru | Wait-Process

This was based on a suggestion made in Moby Issue #30395. The installer completes successfully with Exit code: 0x0, restarting: No, but the container itself hangs for approximately 5 minutes afterwards before the timeout.

The Dockerfile is:

FROM microsoft/iis:windowsservercore-ltsc2016

RUN mkdir C:\content

ADD content/ /content

RUN powershell c:/content/install.ps1

[kzatylny]

I found a workaround that works for me. I run a docker ps -a to see if I have an exited copy of my container sitting around. Then I run a docker rm containerid to remove that exited container and then my docker compose works again from visual studio.

Also, I found that my dockerfile was not updated to use the correct base and build images when I changed my project from using .net core 2.0 to 2.1. I created a new project which targeted .net core 2.1 and copied the FROM lines to my original project and things started to work again.

[kirankumargmrur]

Hi @darstahl I have the same issue when I run docker run from the Windows Insider Program build 17711. I am not able to do docker run for the microsoft/nanoserver-insder image. The following error I got.

docker run --isolation=hyperv -it microsoft/nanoserver-insider cmd

C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: container 16cf36301b02952d8a5befdacf025d996dc93985666483879aa8fef15218c8c9 encountered an error during CreateContainer: hcsshim: timeout waiting for notification extra info: {"SystemType":"Container","Name":"16cf36301b02952d8a5befdacf025d996dc93985666483879aa8fef15218c8c9","Owner":"docker","IgnoreFlushesDuringBoot":true,"LayerFolderPath":"C:\ProgramData\Docker\windowsfilter\16cf36301b02952d8a5befdacf025d996dc93985666483879aa8fef15218c8c9","Layers":[{"ID":"0be3898b-4e57-5b03-9fbe-6de1fec37724","Path":"C:\ProgramData\Docker\windowsfilter\449f1bb3e4f81fa2a6800de27c37066520954c92da260dd14cffd1171a75076e"}],"HostName":"","HvPartition":true,"EndpointList":["],"HvRuntime":{"ImagePath":"C:\ProgramData\Docker\windowsfilter\449f1bb3e4f81fa2a6800de27c37066520954c92da260dd14cffd1171a75076e\UtilityVM"},"AllowUnqualifiedDNSQuery":true}.

[halbherz]

So I had this issue now for a couple of days and couldn't really pin point it down.

What did the trick for me was:

• Uninstalling docker
• Removing all docker folders
• Removing the windows features Containers and Hyper-v
• Restart
• Reinstall docker
• Let docker handle the Windows features
• Restart

And that's it...

[kirankumargmrur]

HI @halbherz Thanks. It worked for me. I want to install a .exe file to nanoserver-insider container Is that possible? if possible how

[shankar1011]

Hi there, i'm also getting same error as "hcsshim: timeout waiting for notification extra info" still unable to figure out the cause of problem and fix. Please comment if you know the way around to fix, thanks.

[Zyril-8204]

Hey there,

I just installed Docker and running it for the first time and I'm getting this error with windows containers

I attempted @halbherz suggestion and was unable to get it working after doing that. Not sure what else to do at this point.

Running Win 10 Enterprise version 1709 os build 16299.98

Running latest vs 2017, .net core 2.1 with docker and brand new project (letting VS do all the docker file stuff).

Any suggestions would be welcome.

[shankar1011]

This issue seems to be temporary and it worked for me when i closed all of the running application on my machine which released few GB of RAM. So, i can conclude it as "This error is coming due to memory issue", Thanks.

[benc-uk]

I've got a 16GB machine, I have nothing else running so have about 10GB ram free and I still get this problem. It's one painful error after another with Windows containers on Windows 10

I'm on Windows 10 1803 on my machine

[Crocmagnon]

I ran into the same issue : hcsshim: timeout waiting for notification extra info I solved it by increasing the RAM usable for a build with the -m option, like so:

docker build -m 4GB -t <tag> .

Replace 4GB with the amount of RAM you need. By default, it's limited to 1GB.

Source: https://sarafian.github.io/sdl/knowledge-center/2017/03/11/docker-windows-containers-out-of-memory-windows-10.html

[Ben-m-s]

This issue prevented one of my projects from progressing this week. I managed to fix it by reinstalling Docker for Windows.

Next are some details about it:

Environment specs

• Docker Version: 18.06.1-ce
• Storage Driver: windowsfilter (windows) lcow (linux)
• Kernel Version: 10.0 17134 (17134.1.amd64fre.rs4_release.180410-1804)
• Operating System: Windows 10 Enterprise Version 1803 (OS Build 17134.285)
• Total Memory: 31.95GiB
• Daemon configuration file:

  {
  "registry-mirrors": [],
  "insecure-registries": [],
  "debug": true,
  "experimental": true,
  "graph": "H:/ProgramData/Docker",
  "storage-opts": [
  "size=20G"
  ]
  }

  Notice that my docker storage folder is not in "C:\". It is in an alternate folder.

Fixing Process

The PowerShell script provided here was used to uninstall Docker for Windows. Please read to the end before trying to use the script.

This is the script:

$ErrorActionPreference = "SilentlyContinue"

kill -force -processname 'Docker for Windows', com.docker.db, vpnkit, com.docker.proxy, com.docker.9pdb, moby-diag-dl, dockerd

try {
    ./MobyLinux.ps1 -Destroy
} Catch {}

$service = Get-WmiObject -Class Win32_Service -Filter "Name='com.docker.service'"
if ($service) { $service.StopService() }
if ($service) { $service.Delete() }
Start-Sleep -s 5
Remove-Item -Recurse -Force "~/AppData/Local/Docker"
Remove-Item -Recurse -Force "~/AppData/Roaming/Docker"
if (Test-Path "C:\ProgramData\Docker") { takeown.exe /F "C:\ProgramData\Docker" /R /A /D Y }
if (Test-Path "C:\ProgramData\Docker") { icacls "C:\ProgramData\Docker\" /T /C /grant Administrators:F }
Remove-Item -Recurse -Force "C:\ProgramData\Docker"
Remove-Item -Recurse -Force "C:\Program Files\Docker"
Remove-Item -Recurse -Force "C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Docker"
Remove-Item -Force "C:\Users\Public\Desktop\Docker for Windows.lnk"
Get-ChildItem HKLM:\software\microsoft\windows\currentversion\uninstall | % {Get-ItemProperty $_.PSPath}  | ? { $_.DisplayName -eq "Docker" } | Remove-Item -Recurse -Force
Get-ChildItem HKLM:\software\classes\installer\products | % {Get-ItemProperty $_.pspath} | ? { $_.ProductName -eq "Docker" } | Remove-Item -Recurse -Force
Get-Item 'HKLM:\software\Docker Inc.' | Remove-Item -Recurse -Force
Get-ItemProperty HKCU:\software\microsoft\windows\currentversion\Run -name "Docker for Windows" | Remove-Item -Recurse -Force
#Get-ItemProperty HKCU:\software\microsoft\windows\currentversion\UFH\SHC | ForEach-Object {Get-ItemProperty $_.PSPath} | Where-Object { $_.ToString().Contains("Docker for Windows.exe") } | Remove-Item -Recurse -Force $_.PSPath
#Get-ItemProperty HKCU:\software\microsoft\windows\currentversion\UFH\SHC | Where-Object { $(Get-ItemPropertyValue $_) -Contains "Docker" }

The PowerShell script "MobyLinux.ps1" was also downloaded from here and executed by the previous one.

The script removed the files from:

• "C:\ProgramData\Docker"
• "C:\Program Files\Docker"
• "C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Docker"

However, the script did not properly uninstall Docker for Windows. It removed most of its files, leaving some files and key entries in the Windows Registry preventing the Docker installation to work afterwards.

After manually removing the damaged "Docker Engine" service and some entries from the Windows Registry, the "Docker for Windows Installer.exe" installation program was able to re-install docker.

Once Docker was installed the original Daemon configuration (displayed above) was restored and, without re-starting the machine, Docker started to work normally using the original images (sored in H:).

I'm unsure what exactly fixed the issue. It may be something the above script did or simply the fact of re-installing Docker for Windows.

I hope it helps.

[Monte-Christo]

Uninstalling 18.0.6 Stable and reinstalling Docker for Windows 2.0.0. beta1 Edge resolved the issue for me (I suspect reinstalling 18.0.6 would have worked as well). Not sure what caused the failure in the first place, but a Windows update had occurred between the time I had used Docker last and the time it broke.

[fujii]

Suddenly, this issue come again to me today. This is the second time for me. First time happened a few months ago, and resolved just by reinstaling Docker. I created a PowerShell script to reproduce:

foreach($i in @(1..10)) {
  docker run --rm --memory=16g webkitdev/msbuild cmd /c echo pass $i
}

Here is the result.

pass 1
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: CreateComputeSystem 8958a4b18e2c257f0e8df7ae4ee4efa16178ded20436ec31e3332d1b923bd1d7: hcsshim: timeout waiting for notification
(extra info: {"SystemType":"Container", (...) ,"AllowUnqualifiedDNSQuery":true}).
pass 3
pass 4
pass 5
pass 6
pass 7
pass 8
pass 9
pass 10

PS C:\work\docker> docker version
Client: Docker Engine - Community
 Version:           18.09.0-ce-beta1
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        78a6bdb
 Built:             Thu Sep  6 22:42:13 2018
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.0-ce-beta1
  API version:      1.39 (minimum version 1.24)
  Go version:       go1.10.3
  Git commit:       78a6bdb
  Built:            Thu Sep  6 22:57:45 2018
  OS/Arch:          windows/amd64
  Experimental:     true

[jimpriest]

Docker updated, threw an error during the update and appeared to uninstall itself. Reinstalled and now I get this message :(

I'm on Windows 10 (1709) I also uninstalled and reinstalled Docker 2.0.0 and get the same error.

Client: Docker Engine - Community Version: 18.09.0 API version: 1.39 Go version: go1.10.4 Git commit: 4d60db4 Built: Wed Nov 7 00:47:51 2018 OS/Arch: windows/amd64 Experimental: false

Server: Docker Engine - Community Engine: Version: 18.09.0 API version: 1.39 (minimum version 1.24) Go version: go1.10.4 Git commit: 4d60db4 Built: Wed Nov 7 00:56:41 2018 OS/Arch: windows/amd64 Experimental: false

[malytskyy]

      Docker updated, threw an error during the update and appeared to uninstall itself.  Reinstalled and now I get this message :(

Client: Docker Engine - Community Version: 18.09.0 API version: 1.39 Go version: go1.10.4 Git commit: 4d60db4 Built: Wed Nov 7 00:47:51 2018 OS/Arch: windows/amd64 Experimental: false Server: Docker Engine - Community Engine: Version: 18.09.0 API version: 1.39 (minimum version 1.24) Go version: go1.10.4 Git commit: 4d60db4 Built: Wed Nov 7 00:56:41 2018 OS/Arch: windows/amd64 Experimental: false

Same here. Unable to work after update. Changing memory limit is not helping

[joostmeijles]

This is my number one issue with Docker for Windows containers.

Is there a timeline for a solution?

[jterry75]

@jhowardmsft - Can you follow up here? It doesn't look like Darren ever got to the bottom of this.

[ducttapecoder-vt]

Seems @AdamRehn

In my testing under Win10, it seems that altering the memory limit for containers between subsequent Docker runs (e.g using -m 6GB for one run and then -m 8GB for the next) can reduce the frequency with which this error occurs. Since Win10 always uses Hyper-V isolation, I'm guessing that changing the memory limit forces Docker to provision a new Hyper-V VM, preventing it from re-using an existing VM that may have fallen victim to whatever condition triggers the timeout.

and @Kuinox

Hi, I alsot get this error, but mostly when the system is out of ram and is swapping.

are on to something here.

I noticed that my issue seems to be related to memory problems as well. My machine was at 5.3GB/8GB and closing applications and dropping to 3GB/8GB didn't help until I swapped -m 4GB to -m 2GB. When it broke again, it didn't fix until I went to -m 3GB. I noticed my FCNS (Finished Creating, Never Started) container issues seems to be related to the vmmem processes popping up with a weird GUID user. If the vmmem process indicated 'suspended' or 0kb used, it was a failure. I also noticed on one occassion that even with -m 2GB, the vmmem was using over 2,500,000Kb. I noticed that no new VMs appeared in the Hyper-V manager when the container was running so there's no way (that I know of) to monitor the progress otherwise. There's a strong correlation to hard faults and this error as well. Even with only 5.3GB/8GB used, I was getting lots of hard faults and the build process was failing for the mssql-server-windows-developer dockerfile. Swapping around the -m parameter seems to stir something and suddenly I was moving again. My machine with 256GB RAM (but no internet) never had this particular error.

Hoping @darstahl or @jhowardmsft can provide some update.

[cowlinator]

I am getting this exact error: docker: Error response from daemon: container ABC encountered an error during CreateContainer: hcsshim: timeout waiting for notification.

I have found a rather inconvenient workaround. (⚠️WARNING: This will delete all of your images and containers!⚠️)

After trying to pull/run the image and it fails, stop all containers and run docker system prune -a. This will delete all of your images AND cached layers. Then, pull the image again. Make sure that none of the layers say layer already exists. Now, when running the image, it succeeds.

Unfortunately, this means that you must wipe your layer cache every time. In my instance, the image is very large with many layers, so this add HOURS to time for running the image. But, it will work consistently.

EDIT: Actually, this workaround does not work consistently, sorry for the misinformation.

[lowenna]

Windows Server 2016 (RS1) or Windows Server 2019 (RS5)? We don't believe this error still occurs on RS5 builds and latest docker versions.

[cowlinator]

Windows 10 v1803

[ducttapecoder-vt]

Windows 10 1809 for me.

On Tue, Mar 5, 2019, 3:54 PM cowlinator notifications@github.com wrote:

Windows 10 v1803

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Microsoft/hcsshim/issues/152#issuecomment-469853059, or mute the thread https://github.com/notifications/unsubscribe-auth/AkMx-oPSOyAGJP1TAxhg10B36BM44ztZks5vTtl7gaJpZM4RJEoL .