Closed MODUSCarstenScholling closed 3 years ago
The reason why the old apps are in the same folder is, that appsourcecop needs them there. I think your fix looks good - I will double check
Thank you Freddy! I mean the baselinePackageCachePath mentioned here: https://docs.microsoft.com/en-us/dynamics365/business-central/dev-itpro/developer/analyzers/appsourcecop-as0003-previousversionnotfound#setting-up-appsourcecop-to-detect-breaking-changes But not sure when this setting was introduced and if this would cause issues with older compiler versions.
Hmm something is still wrong... A bit more context.
Install Apps containing the current dependency apps Install test Apps are the current test dependencies Previous Apps: Previous Detailed Discounts dependencies are satisfied with Foundation/Base passed here:
<Dependencies>
<Dependency Id="ba5e92a6-b5c1-471d-ad51-1f407e627c27" Name="LALA Base" Publisher="LALA" MinVersion="17.0.5.87" CompatibilityId="0.0.0.0" />
<Dependency Id="d78da1d4-e45e-479e-9bb3-c368752fb48f" Name="LALA Foundation" Publisher="LALA" MinVersion="17.0.6.39" CompatibilityId="0.0.0.0" />
</Dependencies>
I would have expected the previous apps to be installed first. Or did I misunderstand the parameters? Because an update means, the dependency version can also be lower.
PARAMETERS
Install Apps
- C:\BcBuild\CiCdStorage\apps\LALA_Foundation_17.0.21030301.0_app.app
- C:\BcBuild\CiCdStorage\apps\LALA_Base_17.2.21030301.0_app.app
Install Test Apps
- C:\BcBuild\CiCdStorage\testApps\LALA_Tests-TestLibraries_17.1.21030301.0_app.app
Previous Apps
- C:\BcBuild\CiCdStorage\oldApps\LALA_Foundation_17.0.21030301.0_app.app
- C:\BcBuild\CiCdStorage\oldApps\LALA_Base_17.1.5.108_app.app
- C:\BcBuild\CiCdStorage\oldApps\LALA_Detailed Discounts_17.1.21030402.21030402_app.app
Application folders
- C:\BuildAgent\_work\22\s\app
Test application folders
- C:\BuildAgent\_work\22\s\test
PULLING GENERIC IMAGE
Pulling generic image took 1 seconds
CREATING CONTAINER
Creating Container master-bld
Version: 17.2.19367.19910-DE
Style: sandbox
Multitenant: Yes
Platform: 17.0.19353.19894
Generic Tag: 1.0.1.6
Container OS Version: 10.0.17763.1879 (ltsc2019)
Host OS Version: 10.0.17763.1879 (ltsc2019)
Using process isolation
Using locale de-DE
INSTALLING APPS
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\5198746c-93e0-4887-9308-4c1acdae3f4a\LALA_Foundation_17.0.21030301.0_app.app
Synchronizing LALA Foundation on tenant default
Installing LALA Foundation on tenant default
App LALA_Foundation_17.0.21030301.0_app.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\fc669520-3338-452d-a438-aff8b9a82405\LALA_Base_17.2.21030301.0_app.app
Synchronizing LALA Base on tenant default
Installing LALA Base on tenant default
App LALA_Base_17.2.21030301.0_app.app successfully published
Installing apps took 41 seconds
IMPORTING TEST TOOLKIT
Importing Test Toolkit for additional country
Synchronizing Test Runner on default
App successfully synchronized
Installing Test Runner on default
App successfully installed
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\2658ac58-b4a6-4d0d-aacc-ea1eaa0b8786\Microsoft_Any_17.2.19367.19910.app
Synchronizing Any on tenant default
Installing Any on tenant default
App Microsoft_Any_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\b0c4f420-0d5f-4daf-b43e-4fe2c5e79ea5\Microsoft_Library Assert_17.2.19367.19910.app
Synchronizing Library Assert on tenant default
Installing Library Assert on tenant default
App Microsoft_Library Assert_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\71553153-b288-4b7c-a2c2-8e78fe0e5d5c\Microsoft_Library Variable Storage_17.2.19367.19910.app
Synchronizing Library Variable Storage on tenant default
Installing Library Variable Storage on tenant default
App Microsoft_Library Variable Storage_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\5a3fa579-21bc-4a19-bf0a-54cbcbea5cd9\Microsoft_System Application Test Library_17.2.19367.19910.app
Synchronizing System Application Test Library on tenant default
Installing System Application Test Library on tenant default
App Microsoft_System Application Test Library_17.2.19367.19910.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\6f8ad01b-e17c-4d42-b52b-9cb7914dd66b\Microsoft_Tests-TestLibraries_17.2.19367.19910.app
Synchronizing Tests-TestLibraries on tenant default
Installing Tests-TestLibraries on tenant default
App Microsoft_Tests-TestLibraries_17.2.19367.19910.app successfully published
TestToolkit successfully imported
Importing Test Toolkit took 27 seconds
INSTALLING TEST APPS
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\49582f06-163c-4fed-9ad9-935bacf9cfab\LALA_Tests-TestLibraries_17.1.21030301.0_app.app
Synchronizing LALA Tests-TestLibraries on tenant default
Installing LALA Tests-TestLibraries on tenant default
App LALA_Tests-TestLibraries_17.1.21030301.0_app.app successfully published
Installing testapps took 2 seconds
COMPILING APPS
Using custom ruleset
Copying previous apps to packages folder
Extracting C:\BuildAgent\_work\22\s\.packages\LALA_Foundation_17.0.21030301.0_app.app
LALA_Foundation = 17.0.21030301.0
Extracting C:\BuildAgent\_work\22\s\.packages\LALA_Base_17.1.5.108_app.app
LALA_Base = 17.1.5.108
Extracting C:\BuildAgent\_work\22\s\.packages\LALA_Detailed Discounts_17.1.21030402.21030402_app.app
LALA_Detailed Discounts = 17.1.21030402.21030402
Creating AppSourceCop.json for validation
Using Symbols Folder: C:\BuildAgent\_work\22\s\.packages
Downloading symbols: Microsoft_Application_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Application&versionText=17.2.0.0&tenant=default
Adding dependency to System Application from Microsoft
Adding dependency to Base Application from Microsoft
Downloading symbols: Microsoft_System_17.0.19353.19894.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=System&versionText=17.0.0.0&tenant=default
Downloading symbols: LALA_Base_17.2.21030301.0.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=MODUS%20Consult%20GmbH&appName=MODUS%20M365%20Base&versionText=17.2.0.0&tenant=default
Downloading symbols: Microsoft_System Application_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=System%20Application&versionText=17.2.0.0&tenant=default
Downloading symbols: Microsoft_Base Application_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Base%20Application&versionText=17.2.0.0&tenant=default
Compiling...
.\alc.exe /project:"c:\sources\app" /packagecachepath:"c:\sources\.packages" /out:"c:\sources\.output\LALA_Detailed Discounts_17.2.21050401.0.app" /analyzer:C:\build\vsix\extension\bin\Analyzers\Microsoft.Dynamics.Nav.CodeCop.dll /analyzer:C:\build\vsix\extension\bin\Analyzers\Microsoft.Dynamics.Nav.AppSourceCop.dll /analyzer:C:\build\vsix\extension\bin\Analyzers\Microsoft.Dynamics.Nav.UICop.dll /ruleset:c:\sources\.vscode\appsource.default.ruleset.json /assemblyprobingpaths:"C:\Program Files (x86)\Microsoft Dynamics NAV\170\RoleTailored Client","C:\Program Files\Microsoft Dynamics NAV\170\Service","C:\Program Files (x86)\Open XML SDK\V2.5\lib","c:\Windows\Microsoft.NET\Assembly","C:\Test Assemblies\Mock Assemblies"
Microsoft (R) AL Compiler version 6.1.5.47101
Copyright (C) Microsoft Corporation. All rights reserved
Compilation started for project 'LALA Detailed Discounts' containing '127' files at '8:18:38.633'.
Compilation ended at '8:18:47.432'.
C:\BuildAgent\_work\22\s\.output\LALA_Detailed Discounts_17.2.21050401.0.app successfully created in 45 seconds
LALA_Detailed Discounts_17.2.21050401.0.app copied to C:\BuildAgent\_work\22\s\.packages
IMPORTING TEST TOOLKIT
Skipping app 'C:\Applications.DE\Microsoft_Test Runner_17.2.19367.19910.app' as it is already installed
Skipping app 'C:\Applications.DE\Microsoft_Any_17.2.19367.19910.app' as it is already installed
Skipping app 'C:\Applications.DE\Microsoft_Library Assert_17.2.19367.19910.app' as it is already installed
Skipping app 'C:\Applications.DE\Microsoft_Library Variable Storage_17.2.19367.19910.app' as it is already installed
Skipping app 'C:\Applications.DE\Microsoft_System Application Test Library_17.2.19367.19910.app' as it is already installed
Skipping app 'C:\Applications.DE\Microsoft_Tests-TestLibraries_17.2.19367.19910.app' as it is already installed
TestToolkit successfully imported
Importing Test Toolkit took 1 seconds
INSTALLING TEST APPS
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\6b398d68-e2e4-4ebd-a378-daef994da896\LALA_Tests-TestLibraries_17.1.21030301.0_app.app
Synchronizing LALA Tests-TestLibraries on tenant default
Installing LALA Tests-TestLibraries on tenant default
WARNUNG: Cannot install extension LALA Tests-TestLibraries by MODUS Consult GmbH 17.1.21030301.0 for the tenant
default because it is already installed.
App LALA_Tests-TestLibraries_17.1.21030301.0_app.app successfully published
Installing testapps took 1 seconds
COMPILING TEST APPS
Using Symbols Folder: C:\BuildAgent\_work\22\s\.packages
Downloading symbols: LALA_Tests-TestLibraries_17.1.21030301.0.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=MODUS%20Consult%20GmbH&appName=MODUS%20M365%20Tests-TestLibraries&versionText=16.0.0.10&tenant=default
Downloading symbols: Microsoft_Tests-TestLibraries_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Tests-TestLibraries&versionText=17.2.0.0&tenant=default
Adding dependency to System Application Test Library from Microsoft
Adding dependency to Library Variable Storage from Microsoft
Downloading symbols: Microsoft_System Application Test Library_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=System%20Application%20Test%20Library&versionText=17.2.0.0&tenant=default
Adding dependency to System Application from Microsoft
Adding dependency to Any from Microsoft
Downloading symbols: Microsoft_Library Variable Storage_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Library%20Variable%20Storage&versionText=17.2.0.0&tenant=default
Adding dependency to Library Assert from Microsoft
Downloading symbols: Microsoft_Any_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Any&versionText=17.2.0.0&tenant=default
Downloading symbols: Microsoft_Library Assert_17.2.19367.19910.app
Url : http://172.29.105.222:7049/BC/dev/packages?publisher=Microsoft&appName=Library%20Assert&versionText=17.2.0.0&tenant=default
Compiling...
.\alc.exe /project:"c:\sources\test" /packagecachepath:"c:\sources\.packages" /out:"c:\sources\.output\LALA_Detailed Discounts Test_17.2.21050401.0.app" /assemblyprobingpaths:"C:\Program Files (x86)\Microsoft Dynamics NAV\170\RoleTailored Client","C:\Program Files\Microsoft Dynamics NAV\170\Service","C:\Program Files (x86)\Open XML SDK\V2.5\lib","c:\Windows\Microsoft.NET\Assembly","C:\Test Assemblies\Mock Assemblies"
Microsoft (R) AL Compiler version 6.1.5.47101
Copyright (C) Microsoft Corporation. All rights reserved
Compilation started for project 'LALA Detailed Discounts Test' containing '3' files at '8:18:53.855'.
Compilation ended at '8:19:1.548'.
C:\BuildAgent\_work\22\s\.output\LALA_Detailed Discounts Test_17.2.21050401.0.app successfully created in 11 seconds
LALA_Detailed Discounts Test_17.2.21050401.0.app copied to C:\BuildAgent\_work\22\s\.packages
Compiling apps, test apps and importing test toolkit took 66 seconds
SIGNING APPS
Copy C:\BcBuild\CiCdStorage\CodeSigningCert.pfx to container master-bld (c:\run\CodeSigningCert.pfx)
Downloading Signing Tools
Installing Signing Tools
Signing c:\sources\.output\LALA_Detailed Discounts_17.2.21050401.0.app
Done Adding Additional Store
Successfully signed: c:\sources\.output\LALA_Detailed Discounts_17.2.21050401.0.app
Signing apps took 11 seconds
INSTALLING PREVIOUS APPS
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\08c38901-3029-4090-99c1-b0694674c2f4\LALA_Foundation_17.0.21030301.0_app.app
Synchronizing LALA Foundation on tenant default
Installing LALA Foundation on tenant default
WARNUNG: Cannot install extension LALA Foundation by MODUS Consult GmbH 17.0.21030301.0 for the tenant default
because it is already installed.
App LALA_Foundation_17.0.21030301.0_app.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\master-bld\6421aedc-5be6-4cdc-984c-48afc8fa548d\LALA_Base_17.1.5.108_app.app
Synchronizing LALA Base on tenant default
Cannot synchronize extension LALA Base because a newer version has already been synchronized.
at <ScriptBlock>, <No file>: line 35
bei Invoke-ScriptInBcContainer, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\ContainerHandling\Invoke-ScriptInNavContainer.ps1: Zeile 44
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Publish-NavContainerApp.ps1: Zeile 257
bei Publish-BcContainerApp, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Publish-NavContainerApp.ps1: Zeile 117
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 506
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 1283
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 1265
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 1260
bei <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 577
bei Run-AlPipeline, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\2.0.11\AppHandling\Run-AlPipeline.ps1: Zeile 544
bei <ScriptBlock>, C:\BcBuild\execute.ps1: Zeile 154
bei <ScriptBlock>, C:\BcBuild\execute.ps1: Zeile 77
bei <ScriptBlock>, C:\BuildAgent\_work\_temp\9fc7d508-5ffe-419c-881c-3cf2aa911a0d.ps1: Zeile 2
bei <ScriptBlock>, <Keine Datei>: Zeile 1
REMOVING CONTAINER
Removing container master-bld
Removing master-bld from container hosts file
Removing master-bld-* from container hosts file
Removing C:\ProgramData\BcContainerHelper\Extensions\master-bld
Removing container took 3 seconds
installApps are installed first - and are ment to be apps you don't control (and also not Microsoft apps that already are in the container) You would not put previousapps in installApps.
Ahh, I understand (hopefully). So there is no concept of previous dependencies, but previous main app only (passed in previousApps). And the previous main app is supposed to work with current dependencies (which must be passed to installApps, because there is no such dependencyApps parameter)!? previousApps only gets the old main app, not any dependencies.
Sounds like you can add them to previousApps - and the new dependencies to apps - they will then also get validated.
That does not work like shown here: https://github.com/microsoft/navcontainerhelper/issues/1906#issuecomment-831729315 Because if installApps has newer versions already (and installed first), previousApps with lower versions cannot be installed later...
Ok, so you have a 3rd party dependency, where the old version of your app (previousApp) doesn't work with the new version of the 3rd party dependency and the new version of your app doesn't work with the old version of the dependency?
If you change 3rd party dependency with own dependency, then yes :)
We have some complex dependency trees between our own and 3rd party apps. In this case, it is all own apps. But with a 2-level dependency. Discounts is dependent on Base and Base is dependent on Foundation. To satisfy dependencies when compiling or doing upgrade checking, these are collected first and then passed into Run-AlPipeline.
We still have breaking changes in structures before we can add these to App Source. OnPrem we can handle these. This is the reason for having old and current dependencies in our concept.
I will put some thought into this scenario and see if I can make a fix for this
Thank you Freddy!
Maybe another parameter called upgradeApps installApps gets installed first. previousApps gets installed compilations happens upgradeapps gets installed/upgraded apps gets installed/upgraded
?
Does the compilation and upgradability check with AppSourceCop need an actual publishing step? Our thought when creating the concept was, that the previous apps never get installed but only needed to let ALC do its job.
I don't need the previous apps to be published if that works and ALC itself can perfectly check the upgradability. If not, I think compilation and upgradeApps must be changed.
In your steps, if the previous apps get installed before compilation, the actual (new) dependencies need to be in the packages folder already. Downloading would download old apps.
An in my opinion working order:
Or another option: dependentApps, but works the same as upgradeApps.
An idea, not sure if this is easily manageable: everytime apps need to be installed, call a script block with the above parameter name, so that changes to the app array can be made :)
But upgradeApps or dependentApps would do. As long as there is a chance to get the current dependencies into the container before compilation happens and to also do an upgrade check.
Yeah - I was just about to write my other suggestion wouldn't work - as you need the latest of the 3rd party dependencies in order to compile - so it wouldn't be an option to install old dependencies first.
The idea of actually installing previousapps is to test the upgrade. If I add a parameter called -skipUpgradeTest, then I could avoid publishing previousApps alltogether and only publish installApps, Apps, Test Apps and then run tests.
That might be a better fix for you. It would still run AppSourceCop - the apps doesn't need to be published for that.
Well, sounds like a good solution. skipUpgradeTest could be a bit misleading, could it be -skipPreviousAppPublishing?
or -doNotPerformUpgrade with this, it will only do appsourcecop validation of upgrade. You can still have upgrade code breaking the upgrade.
If you add the parameter in Run-AlPipeline and change line 1247 to:
if ((!$doNotPerformUpgrade) -and ($previousApps)) {
from
if ($previousApps) {
then you can try it out.
.Parameter doNotPerformUpgrade Include this switch to indicate that you do not want to perform the upgrade. This means that the previousApps are never actually published to the container.
Let me check it. On my way...
It works, great, thanks Freddy! Still using the first fix in compile, right?
Shipped in 2.0.12
Run-AlPipeline with previousApps -> Compile-AppInNavContainer does not download proper versions/symbols before compiling We are running Run-AlPipeline with previousApps. After finalizing our version handling, our previous dependent app versions are far lower than the currently required versions. When Compile-AppInNavContainer is run and it comes to the symbol download, dependencies are checked against app name and app publisher, but not versions. The previous app dependency versions are to low for the current app. They are found and satisfying the Compile-AppInNavContainer command, but not the compiler.
Scripts used to create container and cause the issue
Relevant output of scripts
Additional context This is our temporary fix in Compile-AppInNavContainer:
But I tend to having the old apps for update checking in a different folder. Would you agree?