search

microsoft / navcontainerhelper

Official Microsoft repository for BcContainerHelper, a PowerShell module, which makes it easier to work with Business Central Containers on Docker.
MIT License
384 stars 247 forks source link

Run-AlPipeline with previousApps -> Compile-AppInNavContainer does not download proper versions/symbols before compiling #1906

Closed MODUSCarstenScholling closed 3 years ago

MODUSCarstenScholling commented 3 years ago

Run-AlPipeline with previousApps -> Compile-AppInNavContainer does not download proper versions/symbols before compiling We are running Run-AlPipeline with previousApps. After finalizing our version handling, our previous dependent app versions are far lower than the currently required versions. When Compile-AppInNavContainer is run and it comes to the symbol download, dependencies are checked against app name and app publisher, but not versions. The previous app dependency versions are to low for the current app. They are found and satisfying the Compile-AppInNavContainer command, but not the compiler.

Scripts used to create container and cause the issue

    Run-AlPipeline @params `
        -pipelineName $config.pipelineName `
        -containerName $containerName `
        -imageName $dockerImageName `
        -useGenericImage $genericImageName `
        -artifact $artifactUrl `
        -tenant 'default' `
        -baseFolder $config.buildSourcesPath `
        -credential $containerCredential `
        -licenseFile $licenseFile `
        -installApps $config.types.app.artifacts `
        -installTestApps $config.types.test.artifacts `
        -previousApps $config.types.app.oldArtifacts `
        -assignPremiumPlan `
        -enableCodeCop `
        -enableAppSourceCop `
        -enableUICop `
        -azureDevOps `
        -buildArtifactFolder $config.artifactStagingPath `
        -createRuntimePackages `
        -rulesetFile $config.ruleSetFilePath;

Additional passed params not affecting this issue.

Relevant output of scripts

BcContainerHelper is version 2.0.11
BcContainerHelper is running as administrator
Host is Microsoft Windows Server 2019 Datacenter - ltsc2019
Docker Client Version is 19.03.2
Docker Server Version is 19.03.2
Fetching all docker images
Fetching all docker volumes
ArtifactUrl and ImageName specified
Image sandbox:17.2.19367.19910-de already exists
Using image sandbox:17.2.19367.19910-de
Creating Container master-bld
Version: 17.2.19367.19910-DE
Style: sandbox
Multitenant: Yes
Platform: 17.0.19353.19894
Generic Tag: 1.0.1.6
Container OS Version: 10.0.17763.1879 (ltsc2019)
Host OS Version: 10.0.17763.1879 (ltsc2019)
Using process isolation
Using locale de-DE
Disabling the standard eventlog dump to container log every 2 seconds (use -dumpEventLog to enable)
Using license file C:\BcBuild\CiCdStorage\Licenses\BC17.flf
Additional Parameters:
--volume "C:\BuildAgent\vsts-agent-win-x64-2.153.1\_work\19\s:c:\sources"
--env customNavSettings=EnableTaskScheduler=False
Files in C:\ProgramData\BcContainerHelper\Extensions\master-bld\my:
- AdditionalOutput.ps1
- license.flf
- MainLoop.ps1
- SetupNavUsers.ps1
- SetupVariables.ps1
- updatehosts.ps1
Creating container master-bld from image sandbox:17.2.19367.19910-de
2692e59cb91bb17d6130f44998db8a3317f296be45b0c2260f14d974ed647de6
Waiting for container master-bld to be ready
Initializing...
Setting host.containerhelper.internal to 172.26.96.1 in container hosts file
Starting Container
Hostname is master-bld
PublicDnsName is master-bld
Using NavUserPassword Authentication
Starting Local SQL Server
Starting Internet Information Server
Creating Self Signed Certificate
Self Signed Certificate Thumbprint E15A1F175CEC6E1B4BEA68D3E47D9FE10FDA5BD3
Modifying Service Tier Config File with Instance Specific Settings
Modifying Service Tier Config File with settings from environment variable
Setting EnableTaskScheduler to False
Starting Service Tier
Registering event sources
Creating DotNetCore Web Server Instance
Enabling Financials User Experience
Enabling rewrite rule: Don't rewrite system files
Enabling rewrite rule: Already have tenant specified
Enabling rewrite rule: Hostname (without port) to tenant
Using license file 'c:\run\my\license.flf'
Import License
Dismounting Tenant
Mounting Tenant
Mounting Database for default on server localhost\SQLEXPRESS with AllowAppDatabaseWrite = False
Sync'ing Tenant
Tenant is Operational
Creating http download site
Setting SA Password and enabling SA
Creating admin as SQL User and add to sysadmin
Creating SUPER user
WARNING: The password that you entered does not meet the minimum requirements. 
It should be at least 8 characters long and contain at least one uppercase 
letter, one lowercase letter, and one number.
Assign Premium plan for ADMIN
Container IP Address: 172.26.104.243
Container Hostname  : master-bld
Container Dns Name  : master-bld
Web Client          : http://master-bld/BC/?tenant=default
Dev. Server         : http://master-bld
Dev. ServerInstance : BC
Dev. Server Tenant  : default
Setting master-bld to 172.26.104.243 in host hosts file
Setting master-bld-default to 172.26.104.243 in host hosts file
Setting master-bld-default to 172.26.104.243 in container hosts file

Files:
http://master-bld:8080/ALLanguage.vsix

Container Total Physical Memory is 16.0Gb
Container Free Physical Memory is 10.4Gb

Initialization took 42 seconds
Ready for connections!
Reading CustomSettings.config from master-bld
Creating Desktop Shortcuts for master-bld
Container master-bld successfully created

[...]

Creating container took 53 seconds

INSTALLING APPS

Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\512f3f7f-6ee7-4231-8691-b669ac05256b\LALA_Foundation_17.0.21030301.0_app.app
Synchronizing LALA Foundation on tenant default
Installing LALA Foundation on tenant default
App LALA_Foundation_17.0.21030301.0_app.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\10bc301f-af7d-4e82-b72e-3012f65e4dac\LALA_Base_17.2.21030301.0_app.app
Synchronizing LALA Base on tenant default
Installing LALA Base on tenant default
App LALA_Base_17.2.21030301.0_app.app successfully published

Installing apps took 38 seconds

IMPORTING TEST TOOLKIT

Importing Test Toolkit for additional country 
Synchronizing Test Runner on default
App successfully synchronized
Installing Test Runner on default
App successfully installed
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\58c66457-074a-4949-8bd6-38cac20dc9e9\Microsoft_Any_17.2.19367.19910.app
Synchronizing Any on tenant default
Installing Any on tenant default
App Microsoft_Any_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\ab4a8417-5489-447d-8399-67f59de4a1fc\Microsoft_Library Assert_17.2.19367.19910.app
Synchronizing Library Assert on tenant default
Installing Library Assert on tenant default
App Microsoft_Library Assert_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\da0f6ef5-51bd-46a5-aa22-689dfdf666c7\Microsoft_Library Variable Storage_17.2.19367.19910.app
Synchronizing Library Variable Storage on tenant default
Installing Library Variable Storage on tenant default
App Microsoft_Library Variable Storage_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\125e9e74-6f2e-4636-9ec1-60a167ad4bda\Microsoft_System Application Test Library_17.2.19367.19910.app
Synchronizing System Application Test Library on tenant default
Installing System Application Test Library on tenant default
App Microsoft_System Application Test Library_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\e02e084c-5524-4c9c-be23-da3d90cd3c4f\Microsoft_Tests-TestLibraries_17.2.19367.19910.app
Synchronizing Tests-TestLibraries on tenant default
Installing Tests-TestLibraries on tenant default
App Microsoft_Tests-TestLibraries_17.2.19367.19910.app successfully published
TestToolkit successfully imported

Importing Test Toolkit took 25 seconds

INSTALLING TEST APPS

Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\51253b07-3c93-42f5-b792-59d4259d4d7f\LALA_Tests-TestLibraries_17.1.21030301.0_app.app
Synchronizing LALA Tests-TestLibraries on tenant default
Installing LALA Tests-TestLibraries on tenant default
App LALA_Tests-TestLibraries_17.1.21030301.0_app.app successfully published

Installing testapps took 2 seconds

COMPILING APPS

Using custom ruleset
Copying previous apps to packages folder
Extracting C:\BuildAgent\vsts-agent-win-x64-2.153.1\_work\19\s\.packages\LALA_Foundation_17.0.21030301.0_app.app
LALA M365 Foundation = 17.0.21030301.0
Extracting C:\BuildAgent\vsts-agent-win-x64-2.153.1\_work\19\s\.packages\LALA_Base_17.1.5.108_app.app
LALA Base = 17.1.5.108
Extracting C:\BuildAgent\vsts-agent-win-x64-2.153.1\_work\19\s\.packages\LALA_Detailed Discounts_17.1.21030402.21030402_app.app
LALA Detailed Discounts = 17.1.21030402.21030402
Creating AppSourceCop.json for validation
Using Symbols Folder: C:\BuildAgent\vsts-agent-win-x64-2.153.1\_work\19\s\.packages
Downloading symbols: Microsoft_Application_17.2.19367.19910.app
Url : http://172.26.102.188:7049/BC/dev/packages?publisher=Microsoft&appName=Application&versionText=17.2.0.0&tenant=default
Adding dependency to System Application from Microsoft
Adding dependency to Base Application from Microsoft
Downloading symbols: Microsoft_System_17.0.19353.19894.app
Url : http://172.26.102.188:7049/BC/dev/packages?publisher=Microsoft&appName=System&versionText=17.0.0.0&tenant=default
Downloading symbols: Microsoft_System Application_17.2.19367.19910.app
Url : http://172.26.102.188:7049/BC/dev/packages?publisher=Microsoft&appName=System%20Application&versionText=17.2.0.0&tenant=default
Downloading symbols: Microsoft_Base Application_17.2.19367.19910.app
Url : http://172.26.102.188:7049/BC/dev/packages?publisher=Microsoft&appName=Base%20Application&versionText=17.2.0.0&tenant=default
Compiling...
.\alc.exe /project:"c:\sources\app" /packagecachepath:"c:\sources\.packages" /out:"c:\sources\.output\LALA_Detailed Discounts_17.2.21050301.0.app" /analyzer:C:\build\vsix\extension\bin\Analyzers\Microsoft.Dynamics.Nav.CodeCop.dll /analyzer:C:\build\vsix\extension\bin\Analyzers\Microsoft.Dynamics.Nav.AppSourceCop.dll /analyzer:C:\build\vsix\extension\bin\Analyzers\Microsoft.Dynamics.Nav.UICop.dll /ruleset:c:\sources\.vscode\appsource.default.ruleset.json /assemblyprobingpaths:"C:\Program Files (x86)\Microsoft Dynamics NAV\170\RoleTailored Client","C:\Program Files\Microsoft Dynamics NAV\170\Service","C:\Program Files (x86)\Open XML SDK\V2.5\lib","c:\Windows\Microsoft.NET\Assembly","C:\Test Assemblies\Mock Assemblies"
Microsoft (R) AL Compiler version 6.1.5.47101
Copyright (C) Microsoft Corporation. All rights reserved

Compilation started for project 'LALA Detailed Discounts' containing '127' files at '16:42:29.556'.

##[error]A package with publisher 'LALA', name 'Base', and a version compatible with '17.2.0.0' could not be found in the package cache folder 'c:\sources\.packages'

Additional context This is our temporary fix in Compile-AppInNavContainer:

    $depidx = 0
    while ($depidx -lt $dependencies.Count) {
        $dependency = $dependencies[$depidx]
        #if ($updateSymbols -or !($existingApps | Where-Object {($_.Name -eq $dependency.name) -and ($_.Name -eq "Application" -or $_.Publisher -eq $dependency.publisher)})) {
        if ($updateSymbols -or !($existingApps | Where-Object {($_.Name -eq $dependency.name) -and ($_.Name -eq "Application" -or (($_.Publisher -eq $dependency.publisher) -and ([System.Version]$_.Version -ge [System.Version]$dependency.version)))})) {
            $publisher = $dependency.publisher

But I tend to having the old apps for update checking in a different folder. Would you agree?

freddydk commented 3 years ago

The reason why the old apps are in the same folder is, that appsourcecop needs them there. I think your fix looks good - I will double check

MODUSCarstenScholling commented 3 years ago

Thank you Freddy! I mean the baselinePackageCachePath mentioned here: https://docs.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/analyzers/appsourcecop-as0003-previousversionnotfound#setting-up-appsourcecop-to-detect-breaking-changes But not sure when this setting was introduced and if this would cause issues with older compiler versions.

MODUSCarstenScholling commented 3 years ago

Hmm something is still wrong... A bit more context.

Install Apps containing the current dependency apps Install test Apps are the current test dependencies Previous Apps: Previous Detailed Discounts dependencies are satisfied with Foundation/Base passed here:

  <Dependencies>
    <Dependency Id="ba5e92a6-b5c1-471d-ad51-1f407e627c27" Name="LALA Base" Publisher="LALA" MinVersion="17.0.5.87" CompatibilityId="0.0.0.0" />
    <Dependency Id="d78da1d4-e45e-479e-9bb3-c368752fb48f" Name="LALA Foundation" Publisher="LALA" MinVersion="17.0.6.39" CompatibilityId="0.0.0.0" />
  </Dependencies>

I would have expected the previous apps to be installed first. Or did I misunderstand the parameters? Because an update means, the dependency version can also be lower.

PARAMETERS

Install Apps
- C:\BcBuild\CiCdStorage\apps\LALA_Foundation_17.0.21030301.0_app.app
- C:\BcBuild\CiCdStorage\apps\LALA_Base_17.2.21030301.0_app.app
Install Test Apps
- C:\BcBuild\CiCdStorage\testApps\LALA_Tests-TestLibraries_17.1.21030301.0_app.app
Previous Apps
- C:\BcBuild\CiCdStorage\oldApps\LALA_Foundation_17.0.21030301.0_app.app
- C:\BcBuild\CiCdStorage\oldApps\LALA_Base_17.1.5.108_app.app
- C:\BcBuild\CiCdStorage\oldApps\LALA_Detailed Discounts_17.1.21030402.21030402_app.app
Application folders
- C:\BuildAgent\_work\22\s\app
Test application folders
- C:\BuildAgent\_work\22\s\test

PULLING GENERIC IMAGE
Pulling generic image took 1 seconds

CREATING CONTAINER
Creating Container master-bld
Version: 17.2.19367.19910-DE
Style: sandbox
Multitenant: Yes
Platform: 17.0.19353.19894
Generic Tag: 1.0.1.6
Container OS Version: 10.0.17763.1879 (ltsc2019)
Host OS Version: 10.0.17763.1879 (ltsc2019)
Using process isolation
Using locale de-DE

INSTALLING APPS

Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\5198746c-93e0-4887-9308-4c1acdae3f4a\LALA_Foundation_17.0.21030301.0_app.app
Synchronizing LALA Foundation on tenant default
Installing LALA Foundation on tenant default
App LALA_Foundation_17.0.21030301.0_app.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\fc669520-3338-452d-a438-aff8b9a82405\LALA_Base_17.2.21030301.0_app.app
Synchronizing LALA Base on tenant default
Installing LALA Base on tenant default
App LALA_Base_17.2.21030301.0_app.app successfully published

Installing apps took 41 seconds

IMPORTING TEST TOOLKIT

Importing Test Toolkit for additional country 
Synchronizing Test Runner on default
App successfully synchronized
Installing Test Runner on default
App successfully installed
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\2658ac58-b4a6-4d0d-aacc-ea1eaa0b8786\Microsoft_Any_17.2.19367.19910.app
Synchronizing Any on tenant default
Installing Any on tenant default
App Microsoft_Any_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\b0c4f420-0d5f-4daf-b43e-4fe2c5e79ea5\Microsoft_Library Assert_17.2.19367.19910.app
Synchronizing Library Assert on tenant default
Installing Library Assert on tenant default
App Microsoft_Library Assert_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\71553153-b288-4b7c-a2c2-8e78fe0e5d5c\Microsoft_Library Variable Storage_17.2.19367.19910.app
Synchronizing Library Variable Storage on tenant default
Installing Library Variable Storage on tenant default
App Microsoft_Library Variable Storage_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\5a3fa579-21bc-4a19-bf0a-54cbcbea5cd9\Microsoft_System Application Test Library_17.2.19367.19910.app
Synchronizing System Application Test Library on tenant default
Installing System Application Test Library on tenant default
App Microsoft_System Application Test Library_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\6f8ad01b-e17c-4d42-b52b-9cb7914dd66b\Microsoft_Tests-TestLibraries_17.2.19367.19910.app
Synchronizing Tests-TestLibraries on tenant default
Installing Tests-TestLibraries on tenant default
App Microsoft_Tests-TestLibraries_17.2.19367.19910.app successfully published
TestToolkit successfully imported

Importing Test Toolkit took 27 seconds

INSTALLING TEST APPS

Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\49582f06-163c-4fed-9ad9-935bacf9cfab\LALA_Tests-TestLibraries_17.1.21030301.0_app.app
Synchronizing LALA Tests-TestLibraries on tenant default
Installing LALA Tests-TestLibraries on tenant default
App LALA_Tests-TestLibraries_17.1.21030301.0_app.app successfully published

Installing testapps took 2 seconds

COMPILING APPS

Using custom ruleset
Copying previous apps to packages folder
Extracting C:\BuildAgent\_work\22\s\.packages\LALA_Foundation_17.0.21030301.0_app.app
LALA_Foundation = 17.0.21030301.0
Extracting C:\BuildAgent\_work\22\s\.packages\LALA_Base_17.1.5.108_app.app
LALA_Base = 17.1.5.108
Extracting C:\BuildAgent\_work\22\s\.packages\LALA_Detailed Discounts_17.1.21030402.21030402_app.app
LALA_Detailed Discounts = 17.1.21030402.21030402
Creating AppSourceCop.json for validation
Using Symbols Folder: C:\BuildAgent\_work\22\s\.packages
Downloading symbols: Microsoft_Application_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Application&versionText=17.2.0.0&tenant=default
Adding dependency to System Application from Microsoft
Adding dependency to Base Application from Microsoft
Downloading symbols: Microsoft_System_17.0.19353.19894.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=System&versionText=17.0.0.0&tenant=default
Downloading symbols: LALA_Base_17.2.21030301.0.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=MODUS%20Consult%20GmbH&appName=MODUS%20M365%20Base&versionText=17.2.0.0&tenant=default
Downloading symbols: Microsoft_System Application_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=System%20Application&versionText=17.2.0.0&tenant=default
Downloading symbols: Microsoft_Base Application_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Base%20Application&versionText=17.2.0.0&tenant=default
Compiling...
.\alc.exe /project:"c:\sources\app" /packagecachepath:"c:\sources\.packages" /out:"c:\sources\.output\LALA_Detailed Discounts_17.2.21050401.0.app" /analyzer:C:\build\vsix\extension\bin\Analyzers\Microsoft.Dynamics.Nav.CodeCop.dll /analyzer:C:\build\vsix\extension\bin\Analyzers\Microsoft.Dynamics.Nav.AppSourceCop.dll /analyzer:C:\build\vsix\extension\bin\Analyzers\Microsoft.Dynamics.Nav.UICop.dll /ruleset:c:\sources\.vscode\appsource.default.ruleset.json /assemblyprobingpaths:"C:\Program Files (x86)\Microsoft Dynamics NAV\170\RoleTailored Client","C:\Program Files\Microsoft Dynamics NAV\170\Service","C:\Program Files (x86)\Open XML SDK\V2.5\lib","c:\Windows\Microsoft.NET\Assembly","C:\Test Assemblies\Mock Assemblies"
Microsoft (R) AL Compiler version 6.1.5.47101
Copyright (C) Microsoft Corporation. All rights reserved

Compilation started for project 'LALA Detailed Discounts' containing '127' files at '8:18:38.633'.

Compilation ended at '8:18:47.432'.

C:\BuildAgent\_work\22\s\.output\LALA_Detailed Discounts_17.2.21050401.0.app successfully created in 45 seconds
LALA_Detailed Discounts_17.2.21050401.0.app copied to C:\BuildAgent\_work\22\s\.packages

IMPORTING TEST TOOLKIT  

Skipping app 'C:\Applications.DE\Microsoft_Test Runner_17.2.19367.19910.app' as it is already installed
Skipping app 'C:\Applications.DE\Microsoft_Any_17.2.19367.19910.app' as it is already installed
Skipping app 'C:\Applications.DE\Microsoft_Library Assert_17.2.19367.19910.app' as it is already installed
Skipping app 'C:\Applications.DE\Microsoft_Library Variable Storage_17.2.19367.19910.app' as it is already installed
Skipping app 'C:\Applications.DE\Microsoft_System Application Test Library_17.2.19367.19910.app' as it is already installed
Skipping app 'C:\Applications.DE\Microsoft_Tests-TestLibraries_17.2.19367.19910.app' as it is already installed
TestToolkit successfully imported

Importing Test Toolkit took 1 seconds

INSTALLING TEST APPS

Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\6b398d68-e2e4-4ebd-a378-daef994da896\LALA_Tests-TestLibraries_17.1.21030301.0_app.app
Synchronizing LALA Tests-TestLibraries on tenant default
Installing LALA Tests-TestLibraries on tenant default
WARNUNG: Cannot install extension LALA Tests-TestLibraries by MODUS Consult GmbH 17.1.21030301.0 for the tenant 
default because it is already installed.
App LALA_Tests-TestLibraries_17.1.21030301.0_app.app successfully published

Installing testapps took 1 seconds

COMPILING TEST APPS

Using Symbols Folder: C:\BuildAgent\_work\22\s\.packages
Downloading symbols: LALA_Tests-TestLibraries_17.1.21030301.0.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=MODUS%20Consult%20GmbH&appName=MODUS%20M365%20Tests-TestLibraries&versionText=16.0.0.10&tenant=default
Downloading symbols: Microsoft_Tests-TestLibraries_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Tests-TestLibraries&versionText=17.2.0.0&tenant=default
Adding dependency to System Application Test Library from Microsoft
Adding dependency to Library Variable Storage from Microsoft
Downloading symbols: Microsoft_System Application Test Library_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=System%20Application%20Test%20Library&versionText=17.2.0.0&tenant=default
Adding dependency to System Application from Microsoft
Adding dependency to Any from Microsoft
Downloading symbols: Microsoft_Library Variable Storage_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Library%20Variable%20Storage&versionText=17.2.0.0&tenant=default
Adding dependency to Library Assert from Microsoft
Downloading symbols: Microsoft_Any_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Any&versionText=17.2.0.0&tenant=default
Downloading symbols: Microsoft_Library Assert_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Library%20Assert&versionText=17.2.0.0&tenant=default
Compiling...
.\alc.exe /project:"c:\sources\test" /packagecachepath:"c:\sources\.packages" /out:"c:\sources\.output\LALA_Detailed Discounts Test_17.2.21050401.0.app" /assemblyprobingpaths:"C:\Program Files (x86)\Microsoft Dynamics NAV\170\RoleTailored Client","C:\Program Files\Microsoft Dynamics NAV\170\Service","C:\Program Files (x86)\Open XML SDK\V2.5\lib","c:\Windows\Microsoft.NET\Assembly","C:\Test Assemblies\Mock Assemblies"
Microsoft (R) AL Compiler version 6.1.5.47101
Copyright (C) Microsoft Corporation. All rights reserved

Compilation started for project 'LALA Detailed Discounts Test' containing '3' files at '8:18:53.855'.

Compilation ended at '8:19:1.548'.

C:\BuildAgent\_work\22\s\.output\LALA_Detailed Discounts Test_17.2.21050401.0.app successfully created in 11 seconds
LALA_Detailed Discounts Test_17.2.21050401.0.app copied to C:\BuildAgent\_work\22\s\.packages

Compiling apps, test apps and importing test toolkit took 66 seconds

SIGNING APPS

Copy C:\BcBuild\CiCdStorage\CodeSigningCert.pfx to container master-bld (c:\run\CodeSigningCert.pfx)
Downloading Signing Tools
Installing Signing Tools
Signing c:\sources\.output\LALA_Detailed Discounts_17.2.21050401.0.app
Done Adding Additional Store
Successfully signed: c:\sources\.output\LALA_Detailed Discounts_17.2.21050401.0.app

Signing apps took 11 seconds

INSTALLING PREVIOUS APPS

Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\08c38901-3029-4090-99c1-b0694674c2f4\LALA_Foundation_17.0.21030301.0_app.app
Synchronizing LALA Foundation on tenant default
Installing LALA Foundation on tenant default
WARNUNG: Cannot install extension LALA Foundation by MODUS Consult GmbH 17.0.21030301.0 for the tenant default 
because it is already installed.
App LALA_Foundation_17.0.21030301.0_app.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\6421aedc-5be6-4cdc-984c-48afc8fa548d\LALA_Base_17.1.5.108_app.app
Synchronizing LALA Base on tenant default
Cannot synchronize extension LALA Base because a newer version has already been synchronized.
at <ScriptBlock>, <No file>: line 35
bei Invoke-ScriptInBcContainer, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\ContainerHandling\Invoke-ScriptInNavContainer.ps1: Zeile 44
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Publish-NavContainerApp.ps1: Zeile 257
bei Publish-BcContainerApp, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Publish-NavContainerApp.ps1: Zeile 117
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 506
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 1283
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 1265
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 1260
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 577
bei Run-AlPipeline, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 544
bei <ScriptBlock>, C:\BcBuild\execute.ps1: Zeile 154
bei <ScriptBlock>, C:\BcBuild\execute.ps1: Zeile 77
bei <ScriptBlock>, C:\BuildAgent\_work\_temp\9fc7d508-5ffe-419c-881c-3cf2aa911a0d.ps1: Zeile 2
bei <ScriptBlock>, <Keine Datei>: Zeile 1

REMOVING CONTAINER

Removing container master-bld
Removing master-bld from container hosts file
Removing master-bld-* from container hosts file
Removing C:\ProgramData\BcContainerHelper\Extensions\master-bld

Removing container took 3 seconds
freddydk commented 3 years ago

installApps are installed first - and are ment to be apps you don't control (and also not Microsoft apps that already are in the container) You would not put previousapps in installApps.

MODUSCarstenScholling commented 3 years ago

Ahh, I understand (hopefully). So there is no concept of previous dependencies, but previous main app only (passed in previousApps). And the previous main app is supposed to work with current dependencies (which must be passed to installApps, because there is no such dependencyApps parameter)!? previousApps only gets the old main app, not any dependencies.

freddydk commented 3 years ago

Sounds like you can add them to previousApps - and the new dependencies to apps - they will then also get validated.

MODUSCarstenScholling commented 3 years ago

That does not work like shown here: https://github.com/microsoft/navcontainerhelper/issues/1906#issuecomment-831729315 Because if installApps has newer versions already (and installed first), previousApps with lower versions cannot be installed later...

freddydk commented 3 years ago

Ok, so you have a 3rd party dependency, where the old version of your app (previousApp) doesn't work with the new version of the 3rd party dependency and the new version of your app doesn't work with the old version of the dependency?

MODUSCarstenScholling commented 3 years ago

If you change 3rd party dependency with own dependency, then yes :)

We have some complex dependency trees between our own and 3rd party apps. In this case, it is all own apps. But with a 2-level dependency. Discounts is dependent on Base and Base is dependent on Foundation. To satisfy dependencies when compiling or doing upgrade checking, these are collected first and then passed into Run-AlPipeline.

We still have breaking changes in structures before we can add these to App Source. OnPrem we can handle these. This is the reason for having old and current dependencies in our concept.

freddydk commented 3 years ago

I will put some thought into this scenario and see if I can make a fix for this

MODUSCarstenScholling commented 3 years ago

Thank you Freddy!

freddydk commented 3 years ago

Maybe another parameter called upgradeApps installApps gets installed first. previousApps gets installed compilations happens upgradeapps gets installed/upgraded apps gets installed/upgraded

?

MODUSCarstenScholling commented 3 years ago

Does the compilation and upgradability check with AppSourceCop need an actual publishing step? Our thought when creating the concept was, that the previous apps never get installed but only needed to let ALC do its job.

I don't need the previous apps to be published if that works and ALC itself can perfectly check the upgradability. If not, I think compilation and upgradeApps must be changed.

In your steps, if the previous apps get installed before compilation, the actual (new) dependencies need to be in the packages folder already. Downloading would download old apps.

An in my opinion working order:

Or another option: dependentApps, but works the same as upgradeApps.

An idea, not sure if this is easily manageable: everytime apps need to be installed, call a script block with the above parameter name, so that changes to the app array can be made :)

But upgradeApps or dependentApps would do. As long as there is a chance to get the current dependencies into the container before compilation happens and to also do an upgrade check.

freddydk commented 3 years ago

Yeah - I was just about to write my other suggestion wouldn't work - as you need the latest of the 3rd party dependencies in order to compile - so it wouldn't be an option to install old dependencies first.

The idea of actually installing previousapps is to test the upgrade. If I add a parameter called -skipUpgradeTest, then I could avoid publishing previousApps alltogether and only publish installApps, Apps, Test Apps and then run tests.

That might be a better fix for you. It would still run AppSourceCop - the apps doesn't need to be published for that.

MODUSCarstenScholling commented 3 years ago

Well, sounds like a good solution. skipUpgradeTest could be a bit misleading, could it be -skipPreviousAppPublishing?

freddydk commented 3 years ago

or -doNotPerformUpgrade with this, it will only do appsourcecop validation of upgrade. You can still have upgrade code breaking the upgrade.

freddydk commented 3 years ago

If you add the parameter in Run-AlPipeline and change line 1247 to:

if ((!$doNotPerformUpgrade) -and ($previousApps)) {

from

if ($previousApps) {

then you can try it out.

freddydk commented 3 years ago

.Parameter doNotPerformUpgrade Include this switch to indicate that you do not want to perform the upgrade. This means that the previousApps are never actually published to the container.

MODUSCarstenScholling commented 3 years ago

Let me check it. On my way...

MODUSCarstenScholling commented 3 years ago

It works, great, thanks Freddy! Still using the first fix in compile, right?

freddydk commented 3 years ago

Shipped in 2.0.12