HodorNV / ALOps

ALOps
59 stars 24 forks source link

Create Docker Install Failed #722

Closed stefanJeady closed 6 months ago

stefanJeady commented 7 months ago

Describe the bug Create Docker Install Failed

the used yaml

name: $(Build.BuildId)

variables:
- name: $(alops-licenseid)
- name: $(bclicense)
- name: $(pfxpass)

trigger:
- PROD

pool: DockerBC

  - task: ALOpsDockerCreate@1
    displayName: 'Create Docker Image'
    inputs:
      artifactversion: '21.0.52534.52761'
      artifacttype: 'Sandbox'
      forcecreateimage: false
      versionselect: 'Closest'
      artifactcountry: 'NZ'
      licensefile: '\bbb.bclicense'
      accept_insider_eula: true

  - task: ALOpsDockerStart@1
    displayName: 'Start Docker'
    inputs:
      docker_pull: false
      licensefile: '\bbb.bclicense'

  - task: ALOpsDockerWait@1
    displayName: 'Wait for Docker to start'
    inputs:
     search_string: 'Ready for connections!'

the output

2024-02-07T23:16:43.8130924Z ##[section]Starting: Create Docker Image
2024-02-07T23:16:43.8207740Z ==============================================================================
2024-02-07T23:16:43.8207869Z Task         : ALOps Docker Create
2024-02-07T23:16:43.8207923Z Description  : Create Docker image from Business Central Artifacts
2024-02-07T23:16:43.8207995Z Version      : 1.462.5150
2024-02-07T23:16:43.8208043Z Author       : Hodor
2024-02-07T23:16:43.8208096Z Help         : Create Docker image based on NAV/BC Artifacts.
2024-02-07T23:16:43.8208159Z ==============================================================================
2024-02-07T23:16:44.2342334Z *** Validate configuration
2024-02-07T23:16:44.2602467Z *** Task Inputs:
2024-02-07T23:16:44.2633026Z 
2024-02-07T23:16:44.2703247Z name                      value                                                                                      
2024-02-07T23:16:44.2705008Z ----                      -----                                                                                      
2024-02-07T23:16:44.2706168Z artifactspecification     Specific                                                                                   
2024-02-07T23:16:44.2707176Z artifactversion           21.0.52534.52761                                                                           
2024-02-07T23:16:44.2708130Z artifacttype              Sandbox                                                                                    
2024-02-07T23:16:44.2709114Z artifactcountry           NZ                                                                                         
2024-02-07T23:16:44.2710166Z versionselect             Closest                                                                                    
2024-02-07T23:16:44.2710370Z artifacthint                                                                                                         
2024-02-07T23:16:44.2711331Z multitenant               False                                                                                      
2024-02-07T23:16:44.2712444Z memory                    8G                                                                                         
2024-02-07T23:16:44.2716278Z imageprefix               myImage                                                                                    
2024-02-07T23:16:44.2718016Z dockerregistry                                                                                                       
2024-02-07T23:16:44.2720389Z dockerusername                                                                                                       
2024-02-07T23:16:44.2721407Z dockerpassword                                                                                                       
2024-02-07T23:16:44.2722405Z storageaccount                                                                                                       
2024-02-07T23:16:44.2723379Z sastoken                                                                                                             
2024-02-07T23:16:44.2724321Z forcecreateimage          False                                                                                      
2024-02-07T23:16:44.2725264Z myscripts                                                                                                            
2024-02-07T23:16:44.2726453Z imagenametemplate         %IMAGE_PREFIX%:%ARTIFACT_TYPE%-%ARTIFACT_VERSION%-%ARTIFACT_COUNTRY%-%OS_VERSION%-%OS_LTSC%
2024-02-07T23:16:44.2727217Z use_generic_image                                                                                                    
2024-02-07T23:16:44.2728175Z licensefile               \bbb.bclicense                                                                             
2024-02-07T23:16:44.2729144Z includetesttoolkit        False                                                                                      
2024-02-07T23:16:44.2730097Z includetestlibrariesonly  False                                                                                      
2024-02-07T23:16:44.2731757Z includetestframeworkonly  False                                                                                      
2024-02-07T23:16:44.2732032Z includeperformancetoolkit False                                                                                      
2024-02-07T23:16:44.2733378Z accept_insider_eula       True                                                                                       
2024-02-07T23:16:44.2733496Z 
2024-02-07T23:16:44.2747385Z 
2024-02-07T23:16:44.2753607Z 
2024-02-07T23:16:44.2838453Z *** For documentation, please visit   : https://www.alops.be/documentation
2024-02-07T23:16:44.2840206Z 
2024-02-07T23:16:46.0061983Z *** ALOps License:
2024-02-07T23:16:46.0083935Z   * Licensed To: Acumen Consulting LTD (Collection License)
2024-02-07T23:16:46.0088440Z 
2024-02-07T23:16:46.0207403Z *** Importing required PS-Functions
2024-02-07T23:16:46.0347613Z *** Install/Update BCContainerHelper [https://github.com/microsoft/navcontainerhelper]
2024-02-07T23:16:46.0376587Z *** Check BcContainerHelper
2024-02-07T23:16:46.1151199Z *** Load BcContainerHelper
2024-02-07T23:16:46.1612572Z BcContainerHelper version 6.0.5
2024-02-07T23:16:46.3803255Z BC.HelperFunctions emits usage statistics telemetry to Microsoft
2024-02-07T23:16:48.6339054Z 
2024-02-07T23:16:48.6339449Z ModuleType Version    Name                                ExportedCommands                                             
2024-02-07T23:16:48.6339919Z ---------- -------    ----                                ----------------                                             
2024-02-07T23:16:48.6340180Z Script     6.0.5      BcContainerHelper                   {Add-FontsToBcContainer, Add-GitToAlProjectFolder, AddTele...
2024-02-07T23:16:48.6340267Z 
2024-02-07T23:16:48.6340315Z 
2024-02-07T23:16:48.6340346Z 
2024-02-07T23:16:48.6343407Z 
2024-02-07T23:16:48.6431605Z *** Get Artifact Url for Version [21.0.52534.52761] / Country [NZ] / Type [Sandbox]: Closest
2024-02-07T23:16:48.6730344Z ##[command]"Get-BCArtifactUrl" 
2024-02-07T23:16:48.6730836Z Name                           Value                                                                                   
2024-02-07T23:16:48.6731182Z ----                           -----                                                                                   
2024-02-07T23:16:48.6731302Z accept_insiderEULA             True                                                                                    
2024-02-07T23:16:48.6731422Z select                         Closest                                                                                 
2024-02-07T23:16:48.6731535Z version                        21.0.52534.52761                                                                        
2024-02-07T23:16:48.6731660Z type                           Sandbox                                                                                 
2024-02-07T23:16:48.6732054Z country                        nz                                                                                      
2024-02-07T23:16:48.6732100Z 
2024-02-07T23:16:48.6732135Z 
2024-02-07T23:16:48.6732184Z 
2024-02-07T23:16:51.9870987Z *** Artifact Url: [https://bcartifacts.azureedge.net/sandbox/21.0.46256.58264/nz]
2024-02-07T23:16:51.9891423Z *** Build new Image based on [https://bcartifacts.azureedge.net/sandbox/21.0.46256.58264/nz]
2024-02-07T23:16:52.0015328Z *** Artifact Type   : [sandbox]
2024-02-07T23:16:52.0024002Z *** Artifact Version: [21.0.46256.58264]
2024-02-07T23:16:52.0031257Z *** Artifact Country: [nz]
2024-02-07T23:16:52.1114509Z *** OS: [2022]
2024-02-07T23:16:54.4703729Z *** OS version: [10.0.20348.2159]
2024-02-07T23:16:54.4724671Z *** Image Template: [%IMAGE_PREFIX%:%ARTIFACT_TYPE%-%ARTIFACT_VERSION%-%ARTIFACT_COUNTRY%-%OS_VERSION%-%OS_LTSC%]
2024-02-07T23:16:54.4820037Z *** ImageName: [myimage:sandbox-21.0.46256.58264-nz-10.0.20348.2159-ltsc2022]
2024-02-07T23:16:54.4889767Z *** Set Pipeline variable $(ALOPS_BC_IMAGE) = 'myimage:sandbox-21.0.46256.58264-nz-10.0.20348.2159-ltsc2022'
2024-02-07T23:16:54.5854609Z Error: No such image: myimage:sandbox-21.0.46256.58264-nz-10.0.20348.2159-ltsc2022
2024-02-07T23:16:54.5918528Z *** No 'MyScripts' specified, skipping parse.
2024-02-07T23:16:54.5937160Z *** Create Image: myimage:sandbox-21.0.46256.58264-nz-10.0.20348.2159-ltsc2022
2024-02-07T23:16:54.5955462Z ##[command]"New-BcImage" 
2024-02-07T23:16:54.5955688Z Name                           Value                                                                                   
2024-02-07T23:16:54.5955917Z ----                           -----                                                                                   
2024-02-07T23:16:54.5956038Z includePerformanceToolkit      False                                                                                   
2024-02-07T23:16:54.5956192Z artifactUrl                    https://bcartifacts.azureedge.net/sandbox/21.0.46256.58264/nz                           
2024-02-07T23:16:54.5956359Z licenseFile                    \bbb.bclicense                                                                          
2024-02-07T23:16:54.5956484Z includeTestToolkit             False                                                                                   
2024-02-07T23:16:54.5956602Z memory                         8G                                                                                      
2024-02-07T23:16:54.5956721Z includeTestFrameworkOnly       False                                                                                   
2024-02-07T23:16:54.5956856Z multitenant                    False                                                                                   
2024-02-07T23:16:54.5957126Z imageName                      myimage:sandbox-21.0.46256.58264-nz-10.0.20348.2159-ltsc2022                            
2024-02-07T23:16:54.5957310Z includeTestLibrariesOnly       False                                                                                   
2024-02-07T23:16:54.5957378Z 
2024-02-07T23:16:54.5957415Z 
2024-02-07T23:16:54.5957481Z 
2024-02-07T23:16:54.8624761Z Downloading artifact /sandbox/21.0.46256.58264/nz
2024-02-07T23:16:54.8908185Z Downloading C:\Users\Administrator\AppData\Local\Temp\c1cd6ba4-046e-4d03-bbf7-74e6b53afcb1.zip
2024-02-07T23:16:54.9014454Z Downloading using WebClient
2024-02-07T23:28:23.3616865Z Unpacking artifact to tmp folder using Expand-Archive
2024-02-07T23:28:32.3192569Z Downloading platform artifact /sandbox/21.0.46256.58264/platform
2024-02-07T23:28:32.3220634Z Downloading C:\Users\Administrator\AppData\Local\Temp\6a893f13-5803-43dd-937b-cdac2d9741f3.zip
2024-02-07T23:28:32.3228806Z Downloading using WebClient
2024-02-07T23:40:43.2518047Z Unpacking artifact to tmp folder using Expand-Archive
2024-02-07T23:41:37.1973920Z Building image myimage:sandbox-21.0.46256.58264-nz-10.0.20348.2159-ltsc2022 based on mcr.microsoft.com/businesscentral:10.0.20348.2159 with https://bcartifacts.azureedge.net/sandbox/21.0.46256.58264/nz
2024-02-07T23:41:37.1989322Z Pulling latest image mcr.microsoft.com/businesscentral:10.0.20348.2159
2024-02-07T23:41:38.0706180Z 10.0.20348.2159: Pulling from businesscentral
2024-02-07T23:41:38.1889550Z Generic Tag: 1.0.2.14
2024-02-07T23:41:38.2953849Z Container OS Version: 10.0.20348.2159 (ltsc2022)
2024-02-07T23:41:38.2961940Z Host OS Version: 10.0.20348.2159 (ltsc2022)
2024-02-07T23:41:38.2984280Z Using process isolation
2024-02-07T23:41:38.3098627Z Using license file \bbb.bclicense
2024-02-07T23:41:38.3107246Z Files in c:\bcartifacts.cache\10w31jgy.fto\my:
2024-02-07T23:41:38.3167873Z Copying Platform Artifacts
2024-02-07T23:41:38.3237801Z c:\bcartifacts.cache\sandbox\21.0.46256.58264\platform
2024-02-07T23:41:41.4358290Z Copying Database
2024-02-07T23:41:41.7410021Z Copying Licensefile
2024-02-07T23:41:41.7518697Z Copying ConfigurationPackages
2024-02-07T23:41:41.7526294Z C:\bcartifacts.cache\sandbox\21.0.46256.58264\nz\ConfigurationPackages
2024-02-07T23:41:41.7714914Z Copying Extensions
2024-02-07T23:41:41.7725684Z C:\bcartifacts.cache\sandbox\21.0.46256.58264\nz\Extensions
2024-02-07T23:41:41.8607805Z Copying Applications.NZ
2024-02-07T23:41:41.8614835Z C:\bcartifacts.cache\sandbox\21.0.46256.58264\nz\Applications.NZ
2024-02-07T23:41:42.0576339Z c:\bcartifacts.cache\10w31jgy.fto
2024-02-07T23:41:52.9265155Z Sending build context to Docker daemon   2.02GB
2024-02-07T23:41:52.9271733Z 
2024-02-07T23:41:52.9811111Z Step 1/6 : FROM mcr.microsoft.com/businesscentral:10.0.20348.2159
2024-02-07T23:41:52.9819083Z  ---> b4dd80be76dc
2024-02-07T23:41:52.9826106Z Step 2/6 : ENV DatabaseServer=localhost DatabaseInstance=SQLEXPRESS DatabaseName=CRONUS IsBcSandbox=Y artifactUrl=https://bcartifacts.azureedge.net/sandbox/21.0.46256.58264/nz filesOnly=False
2024-02-07T23:41:53.1463156Z  ---> Running in ff5e08487eb3
2024-02-07T23:41:53.7380816Z Removing intermediate container ff5e08487eb3
2024-02-07T23:41:53.7390591Z  ---> 1eedd4b92a11
2024-02-07T23:41:53.7397262Z Step 3/6 : COPY my /run/
2024-02-07T23:41:54.0770446Z  ---> e9f85e3d9380
2024-02-07T23:41:54.0788508Z Step 4/6 : COPY NAVDVD /NAVDVD/
2024-02-07T23:42:22.0834590Z  ---> 54714dae5ce2
2024-02-07T23:42:22.0842774Z Step 5/6 : RUN \Run\start.ps1 -installOnly
2024-02-07T23:42:22.1096007Z  ---> Running in bf569748f9c4
2024-02-07T23:42:28.8678366Z c:\run\my folder doesn't exist, creating it
2024-02-07T23:42:29.1509181Z Using installer from C:\Run\210-new
2024-02-07T23:42:29.3418679Z Installing Business Central
2024-02-07T23:42:29.3659752Z Installing from DVD
2024-02-07T23:42:29.3965485Z Starting Local SQL Server
2024-02-07T23:42:31.1675808Z Starting Internet Information Server
2024-02-07T23:42:31.5027136Z Copying Service Tier Files
2024-02-07T23:42:31.5146726Z C:\NAVDVD\ServiceTier\Program Files
2024-02-07T23:42:32.0541568Z C:\NAVDVD\ServiceTier\System64Folder
2024-02-07T23:42:32.0745549Z Copying PowerShell Scripts
2024-02-07T23:42:32.0761963Z C:\NAVDVD\WindowsPowerShellScripts\Cloud\NAVAdministration
2024-02-07T23:42:32.1066549Z C:\NAVDVD\WindowsPowerShellScripts\WebSearch
2024-02-07T23:42:32.1221137Z Copying Web Client Files
2024-02-07T23:42:32.1228559Z C:\NAVDVD\WebClient\Microsoft Dynamics NAV
2024-02-07T23:42:33.6072204Z Copying ModernDev Files
2024-02-07T23:42:33.6080500Z C:\NAVDVD
2024-02-07T23:42:33.6215828Z C:\NAVDVD\ModernDev\program files\Microsoft Dynamics NAV
2024-02-07T23:42:33.7550249Z Copying additional files
2024-02-07T23:42:33.7643958Z Copying ConfigurationPackages
2024-02-07T23:42:33.7651940Z C:\NAVDVD\ConfigurationPackages
2024-02-07T23:42:33.7840182Z Copying Test Assemblies
2024-02-07T23:42:33.7848022Z C:\NAVDVD\Test Assemblies
2024-02-07T23:42:33.8043829Z Copying Extensions
2024-02-07T23:42:33.8051193Z C:\NAVDVD\Extensions
2024-02-07T23:42:33.8922734Z Copying Applications
2024-02-07T23:42:33.8929969Z C:\NAVDVD\Applications
2024-02-07T23:42:34.1791291Z Copying Applications.NZ
2024-02-07T23:42:34.1798614Z C:\NAVDVD\Applications.NZ
2024-02-07T23:42:34.2596823Z Copying dependencies
2024-02-07T23:42:34.2697677Z Copying ReportBuilder
2024-02-07T23:42:47.0706802Z Importing PowerShell Modules
2024-02-07T23:42:59.6753510Z Restoring CRONUS Demo Database
2024-02-07T23:43:04.3748734Z Setting CompatibilityLevel for CRONUS on localhost\SQLEXPRESS
2024-02-07T23:43:04.8940129Z Modifying Business Central Service Tier Config File for Docker
2024-02-07T23:43:04.9528821Z Creating Business Central Service Tier
2024-02-07T23:43:05.0975195Z Installing SIP crypto provider: 'C:\Windows\System32\NavSip.dll'
2024-02-07T23:43:05.3092717Z Starting Business Central Service Tier
2024-02-07T23:45:30.5494352Z Importing CRONUS license file
2024-02-07T23:45:31.1829781Z The server "BC" was unable to process the request. The application will close.
2024-02-07T23:45:31.2332372Z Installation failed
2024-02-07T23:45:31.2337572Z At C:\Run\start.ps1:393 char:9
2024-02-07T23:45:31.2345078Z +         throw "Installation failed"
2024-02-07T23:45:31.2352281Z +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~
2024-02-07T23:45:31.2359824Z     + CategoryInfo          : OperationStopped: (Installation failed:String) [ 
2024-02-07T23:45:31.2366785Z    ], RuntimeException
2024-02-07T23:45:31.2374167Z     + FullyQualifiedErrorId : Installation failed
2024-02-07T23:45:31.2381453Z  
2024-02-07T23:45:47.4917081Z 
2024-02-07T23:45:50.1782731Z BcContainerHelper\New-BcImage Telemetry Correlation Id: 7d8379b6-96e4-40d5-b4d2-3540a027b13b
2024-02-07T23:45:50.2999990Z ##[error]The command 'powershell -Command $ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue'; \Run\start.ps1 -installOnly' returned a non-zero code: 1
ExitCode: 1
Commandline: docker build --isolation=process --memory 8G --no-cache --tag myimage:sandbox-21.0.46256.58264-nz-10.0.20348.2159-ltsc2022 c:\bcartifacts.cache\10w31jgy.fto
2024-02-07T23:45:50.3220550Z ##[section]Finishing: Create Docker Image

Expected behavior Install without error after install CRONUS.

Additional context Using the latest version of BCCH.

waldo1001 commented 7 months ago

I think BCCH has a problem processing the license like this.

licensefile: '\bbb.bclicense'

Can you try either:

stefanJeady commented 7 months ago

Removing the license file from this step allowed it to continue. The path was already the full path as the file was in working dir, before the last few runs this always worked. Using blob storage with a direct SAS URI I got the error could not resolve accountnamehere.windows.net

Also likely related but technically another issue that started at the same time as this:

It looks like something changed with BC 23.2 all our pipelines above this version with the same ymal and comit version as pervious suscessful runs now fail on compile app reading settings .json (there is no issue reading app.json a few lines prior)

If I run the same pipeline with 23.0 it works without issue.


*** Loading Json Object: [c:\Run\DevOps\APS_Core_BC\.vscode\settings.json]
##[error]Exception calling ".ctor" with "1" argument(s): "Invalid URI: The format of the URI could not be determined."
Async Command Start: Add Build Tag
Build '399' has following tags now: AL.Compiler.ALLanguage, BC.Platform.23.0.15817.0, BC.Version.23.4.15643.15915, myimage-sandbox-23.4.15643.15915-nz-10.0.20348.2227-ltsc2022
Async Command End: Add Build Tag```
JeremyVyska commented 7 months ago

@waldo1001 - I'm running into the same Exception problem:

` VSCode Settings: [c:\Run\DevOps\SpareBrainedLicensing_APP.vscode\settings.json] VSCode-Settings json found. *** Loading Json Object: [c:\Run\DevOps\SpareBrainedLicensing_APP.vscode\settings.json]

[error]Exception calling ".ctor" with "1" argument(s): "Invalid URI: The format of the URI could not be determined."

Async Command Start: Add Build Tag `

This only started happening when I added a relative path ruleset to the settings.json.

We've tried both:

"al.ruleSetPath": "src\\.rulesets\\SPBLicensing.ruleset.json"

and

"al.ruleSetPath": "src/.rulesets/SPBLicensing.ruleset.json"

This src folder is in repo/[AppFolderName]/src so I'm wondering if it's an issue with it being in a subfolder. The weird thing is, I have a different app still built against v22 with the same folder structure and ruleset handling. This one I'm building now is against v23 latest.

JanBoes commented 7 months ago

We have the same issue. After removing the settings.json file everything worked fine. And we also use relative path to our ruleset as JermemyVyska is mentioning.

settings.json:

{
    "al.ruleSetPath": "MBS.ruleset.json",
    "al.codeAnalyzers": [
        "${CodeCop}",
        "${UICop}"
    ],
    "CRS.FileNamePattern": "<ObjectNameShort>.<ObjectTypeShortPascalCase>.al",
    "CRS.FileNamePatternExtensions": "<ObjectNameShort>.<ObjectTypeShortPascalCase>.al",
    "CRS.FileNamePatternPageCustomizations": "<ObjectNameShort>.<ObjectTypeShortPascalCase>.al",
    "CRS.OnSaveAlFileAction": "Rename",
    "CRS.ObjectNamePrefix": "INB ",
    "CRS.RemovePrefixFromFilename": true,
    "alOutline.fixCodeCopMissingParenthesesOnSave": true          
}
stefanJeady commented 7 months ago

@waldo1001 Is there an update for this?

waldo1001 commented 7 months ago

We didn't have time to look into this.

Doesn't really sound like a blocking issue as it should be possible to work with a full path / download link (the latter being best practice anyway)?

JanBoes commented 7 months ago

Hi @waldo1001

Sorry but I find a blocking issue as we have to remove our settings.json to be able to run a successful pipeline.

waldo1001 commented 7 months ago

Ok, it seems I misunderstood .. sorry.

Can anyone send me a repro (a (simple) yaml that fails) for the ruleset-issue? That seems ALOps-related.

waldo1001 commented 7 months ago

yaml: image subfolder ruleset: image settings.json: image Using ruleset: image

So I need that repro, I'm afraid .. it's a bit searching in the dark now ..

JeremyVyska commented 7 months ago

I wonder if the ./ at the start is the secret sauce there, or if it's specifically only with multi-root apps.

We had tried "al.ruleSetPath": "src/.rulesets/MainThing.ruleset.json"

In our case, the repo has

MainThing_APP MainThing_TEST

as two apps in the base of the repo. Within the MainThing_APP, we have the folders like

[Repo]/MainThing_APP/src/.rulesets/MainThing.ruleset.json

So, what works fine in VSC is that the [Repo]/MainThing_APP/.vscode/settings.json has the path as relative to the MainThing_APP/ root, rather than the repo root. My hunch is that is the distinction.

I can try to whip up a simple repro-repo tomorrow?

waldo1001 commented 7 months ago

We also have 2 apps: App & Test.

Rulesets are here: App/.codeAnalysis/app.ruleset.json Test/.codeAnalysis/app.ruleset.json

Settings are here: App/.vscode/settings.json Test/.vscode/settings.json

So pretty similar to yours.

Would be awesome to receive a repro

JeremyVyska commented 7 months ago

OK, just was testing this live with @alphakintari :

Yep. it's all pathing issues.

To have a 'relative' to the app path, you have to start with ./

So, for example, if you had:

src/.rulesets/ALOpsIssue722.ruleset.json

This works fine in VSC, but will cause an error in ALOps pipelines.

We updated it to:

./src/.rulesets/ALOpsIssue722.ruleset.json

And it powered right through. So, relative-to-the-app can work, but you have to give it that specific "root hint" at the start.

waldo1001 commented 7 months ago

@stefanJeady , did you try ´./bbb.bclicense´

waldo1001 commented 6 months ago

@stefanJeady , I'll close the issue. Please reopen when still not working for you.