Container creation error: "driver "windowsfilter" failed to remove root filesystem: failed to detach VHD: The device is not ready"

[ninten]

We use an agent to create a container, install some apps, execute tests and remove the container. After some runs, the agent stops at the beginning of the container creation, showing the infamous error: "driver "windowsfilter" failed to remove root filesystem: failed to detach VHD: The device is not ready".

This is how we create the container:

$Credential = [PSCredential]::new('username',(ConvertTo-SecureString 'password' -AsPlainText -Force));
New-BCContainer -accept_eula -assignPremiumPlan -accept_outdated -memoryLimit 12G -doNotCheckHealth -containerName 'build' -credential $Credential -image mcr.microsoft.com/businesscentral/sandbox:16.2.13509.13794-w1-ltsc2019 -restart no -shortcuts None -licenseFile 'C:\xyz\xyzLicense.flf' -includeTestToolkit -includeTestLibrariesOnly -isolation hyperv  -bakFile 'C:\ProgramData\BCContainerHelper\Extensions\Backup\database.bak' 

And this is how we remove it:

Stop-NavContainer -containerName 'build'
Remove-BCContainer -containerName 'build'; Flush-ContainerHelperCache

The stack:

BcContainerHelper is version 1.0.7
BcContainerHelper is running as administrator
Host is Microsoft Windows 10 Pro - ltsc2019
Docker Client Version is 19.03.13
Docker Server Version is 19.03.13
Fetching all docker images
WARNING: You are running specific Docker images from mcr.microsoft.com. These images will no longer be updated, you should switch to user Docker artifacts. See https://freddysblog.com/2020/07/05/july-updates-are-out-they-are-the-last-on-premises-docker-images/
Using image mcr.microsoft.com/businesscentral/sandbox:16.2.13509.13794-w1-ltsc2019
Disabling Health Check (always report healthy)
Creating Container build5720
Version: 16.2.13509.13794-w1
Style: sandbox
Multitenant: No
Platform: 16.0.13440.13772
Generic Tag: 0.0.9.99
Container OS Version: 10.0.17763.973 (ltsc2019)
Host OS Version: 10.0.17763.107 (ltsc2019)
Using hyperv isolation
Using locale en-US
Disabling the standard eventlog dump to container log every 2 seconds (use -dumpEventLog to enable)
Using license file C:\agent\_work\_temp\KuragoLicense16.flf
Files in C:\ProgramData\BcContainerHelper\Extensions\build5720\my:
- AdditionalOutput.ps1
- license.flf
- MainLoop.ps1
- SetupNavUsers.ps1
- SetupVariables.ps1
- updatehosts.ps1
Creating container build5720 from image mcr.microsoft.com/businesscentral/sandbox:16.2.13509.13794-w1-ltsc2019
6cc28f2cfeb277d59844d25ff3527a9791384e0c9fe4b56b0787cf0e3479f76a
##[error]Docker : Error response from daemon: container 6cc28f2cfeb277d59844d25ff3527a9791384e0c9fe4b56b0787cf0e3479f76a: 
driver "windowsfilter" failed to remove root filesystem: failed to detach VHD: The device is not ready.: rename 
C:\ProgramData\Docker\windowsfilter\6cc28f2cfeb277d59844d25ff3527a9791384e0c9fe4b56b0787cf0e3479f76a 
C:\ProgramData\Docker\windowsfilter\6cc28f2cfeb277d59844d25ff3527a9791384e0c9fe4b56b0787cf0e3479f76a-removing: Access 

• We tried using different names for the container on each run (adding the build number) and we still get the error. Even if the removal of the previous container is unsuccessful, the new one shouldn't be affected, right?
• We tried disabling the antivirus as we were told in a previous error.

[freddydk]

This seems to be running on an un-updated Windows Server 2019 - the RTM version. You should always make sure you are running on the latest host OS. And you should change from images to artifacts to make sure that your container uses an OS, which is compatible with the host. I cannot promise you that this solves the issue, but incompatibility between host and container OS has left to many many hours of troubleshooting without any good result.

This is the primary reason why we stopped shipping images and changed to artifacts. https://freddysblog.com/2020/06/25/working-with-artifacts/

[inigo666]

Hi, Freddy. Ninten´s host is Microsoft Windows 10 Pro - ltsc2019 It's a W10 with Docker Desktop for Windows.

Another question, in order to get more availability, and to get a reliable and stable environment, do you recommend to use W10 or W2019 with Docker and BC?

Thanks

[freddydk]

Correct, but it is actually wrong to call it ltsc2019 (as I do in the output), it is 1809 (ltsc is long term servicing channel and ONLY Windows Server - I overlooked the 3rd line). He is running an un-updated Windows 10 from october 2018 (the RTM version). He should get the latest updates of 1809 and furthermore, 1809 will go out of support on November 10th this year.

I think Windows Server is more stable but the real enemy of Docker is 3rd party software, antivirus or security policies enforced on computers from IT departments, who doesn't know anything about Docker.

http://aka.ms/getbc will create an Azure VM with Windows Server 2019 - and it just always works.

[inigo666]

Hi Freddy,

About the template you sent us, each time we execute it, it creates a new network and we cannot reach it, is there any way to modify the template to add this new machine to an existing virtual network?

Thanks

[freddydk]

It has a hardcoded networkname (and reuses that). You could rename your network or you can clone the https://github.com/microsoft/nav-arm-templates repo and modify the getbc.json arm template to match your needs.

[inigo666]

Hi again

If we use our vnet config, it answers us with a bad request error

Instead of try to use our existin gnetwork it tries to delete it. How could we use our actual network?

{ "error": { "code": "InUseSubnetCannotBeDeleted", "message": "Subnet GatewaySubnet is in use by /subscriptions/cb62a524-f468-4552-b201-8add0a509e00/resourceGroups/Integration/providers/Microsoft.Network/virtualNetworkGateways/Integration-VPN-GW/ipConfigurations/default and cannot be deleted. In order to delete the subnet, delete all the resources within the subnet. See aka.ms/deletesubnet.", "details": [] } }

[freddydk]

It is hard for me to say when I don't know your setup. But basically this is just running an Azure VM of type Windows Server 2019 with containers - and then install bccontainerhelper.