search

HodorNV / ALOps

ALOps
55 stars 24 forks source link

ALOpsDockerCreate - hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3) #753

Closed teddy515 closed 1 month ago

teddy515 commented 2 months ago

We have two different build agent that are hosted on two different Microsoft Windows Server 2022 Standard, Version 10.0.20348 Build 20348.

PS C:\Windows\system32> docker images
REPOSITORY                          TAG                                            IMAGE ID       CREATED       SIZE
myimage                             onprem-23.2.14098.14562-dk-ltsc2022-ltsc2022   1939b21a76e0   6 hours ago   9.8GB
mcr.microsoft.com/businesscentral   ltsc2022                                       414f65e50c6c   11 days ago   7.75GB

PS C:\Windows\system32> docker info
Client:
 Version:    26.0.1
 Context:    default
 Debug Mode: false

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 4
 Server Version: 26.0.1
 Storage Driver: windowsfilter
  Windows: 
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local splunk syslog
 Swarm: inactive
 Default Isolation: process
 Kernel Version: 10.0 20348 (20348.1.amd64fre.fe_release.210507-1500)
 Operating System: Microsoft Windows Server Version 21H2 (OS Build 20348.2402)
 OSType: windows
 Architecture: x86_64
 CPUs: 4
 Total Memory: 16GiB
 Name: xxxx
 ID: xxxx
 Docker Root Dir: C:\ProgramData\Docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

One of our agents is working fine when trying to build the pipeline, but the other one stopped working after 9th April.

I have tried to reinstall docker, and also tried to run both this script and the script from freddysblog, with some modification for our 2022 version:

## Flush BCContainerHelper cache, including artifacts
Flush-ContainerHelperCache -keepDays 7
## Delete all unused images and containers
docker system prune -a -f

## Pull the latest BC image
docker pull (Get-BestGenericImageName)

## Restart docker services
Restart-Service *docker*

the used yaml that is failing in our build

  # Initialize Docker
  - task: ALOpsDockerCreate@1
    displayName: 'Create Docker Image' # It will reuse the image if already exists, or build one if it doesn't.
    inputs:
      artifactversion: '$(artifactversion)'
      artifacttype: OnPrem
      artifactcountry: 'DK'
      memory: '16G'

the output

Task         : ALOps Docker Create
Description  : Create Docker image from Business Central Artifacts
Version      : 1.464.6041
Author       : Hodor
Help         : Create Docker image based on NAV/BC Artifacts.
==============================================================================
*** Validate configuration
*** Task Inputs:

name                      value                                                                                      
----                      -----                                                                                      
artifactspecification     Specific                                                                                   
artifactversion           23.2.14098.14562                                                                           
artifacttype              OnPrem                                                                                     
artifactcountry           DK                                                                                         
versionselect             Latest                                                                                     
artifacthint                                                                                                         
multitenant               False                                                                                      
memory                    16G                                                                                        
imageprefix               myImage                                                                                    
dockerregistry                                                                                                       
dockerusername                                                                                                       
dockerpassword                                                                                                       
storageaccount                                                                                                       
sastoken                                                                                                             
forcecreateimage          False                                                                                      
myscripts                                                                                                            
imagenametemplate         %IMAGE_PREFIX%:%ARTIFACT_TYPE%-%ARTIFACT_VERSION%-%ARTIFACT_COUNTRY%-%OS_VERSION%-%OS_LTSC%
use_generic_image                                                                                                    
licensefile                                                                                                          
includetesttoolkit        False                                                                                      
includetestlibrariesonly  False                                                                                      
includetestframeworkonly  False                                                                                      
includeperformancetoolkit False                                                                                      
accept_insider_eula       False                                                                                      
True

*** Importing required PS-Functions
*** Install/Update BCContainerHelper [https://github.com/microsoft/navcontainerhelper]
*** Check BcContainerHelper
*** Install BcContainerHelper
*** Load BcContainerHelper
BcContainerHelper version 6.0.15
BC.HelperFunctions emits usage statistics telemetry to Microsoft
Running on Windows, PowerShell 5.1.20348.2400

ModuleType Version    Name                                ExportedCommands                                             
---------- -------    ----                                ----------------                                             
Script     6.0.15     BcContainerHelper                   {Add-FontsToBcContainer, Add-GitToAlProjectFolder, AddTele...

*** Get Artifact Url for Version [23.2.14098.14562] / Country [DK] / Type [OnPrem]: Latest
"Get-BCArtifactUrl" 
Name                           Value                                                                                   
----                           -----                                                                                   
select                         Latest                                                                                  
version                        23.2.14098.14562                                                                        
type                           OnPrem                                                                                  
country                        dk            

 *** Artifact Url: [https://bcartifacts-exdbf9fwegejdqak.b02.azurefd.net/onprem/23.2.14098.14562/dk]
*** Build new Image based on [https://bcartifacts-exdbf9fwegejdqak.b02.azurefd.net/onprem/23.2.14098.14562/dk]
*** Artifact Type   : [onprem]
*** Artifact Version: [23.2.14098.14562]
includeTestToolkit             False                                                                                   
memory                         16G                                                                                     
artifactUrl                    https://bcartifacts-exdbf9fwegejdqak.b02.azurefd.net/onprem/23.2.14098.14562/dk         
multitenant                    False                                                                                   
imageName                      myimage:onprem-23.2.14098.14562-dk-ltsc2022-ltsc2022                                    
includeTestLibrariesOnly       False                                                                                   
includeTestFrameworkOnly       False                                                                                   

Downloading artifact /onprem/23.2.14098.14562/dk
Downloading C:\Users\xxx\AppData\Local\Temp\45512979-3482-41c1-b1fd-cbeb70641bc0.zip
Downloading using WebClient
Unpacking artifact to tmp folder using Expand-Archive
Downloading platform artifact /onprem/23.2.14098.14562/platform
Downloading C:\Users\xxx\AppData\Local\Temp\ef407b14-27bc-4ada-b848-607a4d838a6d.zip
Downloading using WebClient
Unpacking artifact to tmp folder using Expand-Archive
Downloading Prerequisite Components
Downloading c:\bcartifacts.cache\onprem\23.2.14098.14562\platform\Prerequisite Components\DotNetCore\DotNetCore.1.0.4_1.1.1-WindowsHosting.exe
Downloading using WebClient
Downloading c:\bcartifacts.cache\onprem\23.2.14098.14562\platform\Prerequisite Components\IIS URL Rewrite Module\rewrite_2.0_rtw_x64.msi
Downloading using WebClient
Building image myimage:onprem-23.2.14098.14562-dk-ltsc2022-ltsc2022 based on mcr.microsoft.com/businesscentral:ltsc2022 with https://bcartifacts-exdbf9fwegejdqak.b02.azurefd.net/onprem/23.2.14098.14562/dk
Pulling latest image mcr.microsoft.com/businesscentral:ltsc2022
ltsc2022: Pulling from businesscentral

...

BcContainerHelper\New-BcImage Telemetry Correlation Id: xxxx
**##[error]re-exec error: exit status 1: output: hcsshim::ImportLayer failed in Win32: The system cannot find the path specified. (0x3)
ExitCode: 1**
Commandline: docker build --isolation=process --memory 16G --no-cache --tag myimage:onprem-23.2.14098.14562-dk-ltsc2022-ltsc2022 c:\bcartifacts.cache\4nrnk10a.s0f
Finishing: Create Docker Image

Expected behavior That a Docker container is created with a bccontainer that we can run our automated tests inside.

Other info In the other server, we are fetching an already existing image, which makes the build successful...

*** Artifact Url: [https://bcartifacts-exdbf9fwegejdqak.b02.azurefd.net/onprem/23.2.14098.14562/dk]
*** Build new Image based on [https://bcartifacts-exdbf9fwegejdqak.b02.azurefd.net/onprem/23.2.14098.14562/dk]
*** Artifact Type   : [onprem]
*** Artifact Version: [23.2.14098.14562]
*** Artifact Country: [dk]
*** OS: [2022]
*** OS version: [ltsc2022]
*** Image Template: [%IMAGE_PREFIX%:%ARTIFACT_TYPE%-%ARTIFACT_VERSION%-%ARTIFACT_COUNTRY%-%OS_VERSION%-%OS_LTSC%]
*** ImageName: [myimage:onprem-23.2.14098.14562-dk-ltsc2022-ltsc2022]
*** Set Pipeline variable $(ALOPS_BC_IMAGE) = 'myimage:onprem-23.2.14098.14562-dk-ltsc2022-ltsc2022'
*** Image already locally cached. Skipping Pull/Create image.
*** Docker Create Succeeded.
*** Cleanup VSTS Environment: True
waldo1001 commented 2 months ago

Could you

  1. try with "forcecreateimage" to true?
  2. test to create the image with BCCH on the build agent? Just to see what happens if you do that? If it fails, you'd be able to also report this on the BCCH forum - may be they know more?
epernst commented 2 months ago

We have a developer with the same error, when building a local sandbox using BCCH. So not an ALOps issue!

waldo1001 commented 2 months ago

@teddy515 , do you agree?

waldo1001 commented 1 month ago

I'll close this issue. Reopen if necessary.