"Not Found" error on BC22 pipeline when compiling test app

lvanvugt commented 1 year ago

Issue

Building my app and test app on BC22 throws an erro when compiling the test app and wanting to download the dependencies. On BC21, 20, 19 and 18 I have no issues at all using the same yaml.

The error occurs when wanting to download Any app. For full log see below, but this is the section with the error:

*** Download Application-Symbols for [Any v18.0.0.0]
*** Downloading NAV Symbols for [Any v18.0.0.0]
*** Downloading NAV Symbols from [https://6026cb1916d4:7049/BC/dev/packages?publisher=Microsoft&appName=Any&versionText=18.0.0.0&tenant=default&appid=e7320ebb-08b3-4406-b1ec-b4927d3e280b]
*** Downloading NAV Symbols with BasicAuthentication [admin]
Invoke-RestMethod -Method Get -Uri https://6026cb1916d4:7049/BC/dev/packages?publisher=Microsoft&appName=Any&versionText=18.0.0.0&tenant=default&appid=e7320ebb-08b3-4406-b1ec-b4927d3e280b
##[error]Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (404) Not Found."

I reran the pipeline to let it create a SQL backup that I restored and indeed all test library apps, like Any, Library Assert, Tests-Test Libraries, etc. have not been deployed into this database.

Used yaml

name: $(Build.BuildId)

trigger:
 branches:
   include:
     - master/LookupValue-Extension

variables:
- group: 'PipelineVariables'

pool: default

steps:
- checkout: self
  clean: true 

- task: ALOpsDockerCreate@1
  displayName: 'ALOPS - Create Docker Image' # It will reuse the image if already exists, or build one if it doesn't.
  inputs:
    artifactversion: '$(artifactVersion)'
    artifactcountry: '$(artifactCountry)'
    forcecreateimage: $(forceCreateImage)
    includetesttoolkit: true
    includetestlibrariesonly: true

- task: ALOpsDockerStart@1
  displayName: 'ALOPS - Start Docker Container' # No need to provide any details - it will get the details from previous step
  inputs:
    enable_api_services: true

- task: ALOpsDockerWait@1
  displayName: 'ALOPS - Wait for Docker Container to start'
  inputs:
    search_string: 'Ready for connections!'

- task: ALOpsLicenseImport@1
  displayName: 'ALOPS - License Import'
  inputs:
    usedocker: true
    license_path: '$(LuxBeheerBcDevLicense)'

- task: ALOpsAppCompiler@1
  displayName: 'ALOPS - Compile Extension: App'
  inputs:
    usedocker: true
    nav_app_version: '?.?.[yyyyWW].*'
    targetproject: 'LookupValue Extension/App/app.json'
    app_file_suffix: '_APP'                                 # A trick to uniquely identify the APP (from the TEST-app)

- task: ALOpsAppPublish@1
  displayName: 'ALOPS - Publish Extension'
  inputs:
    usedocker: true
    nav_artifact_app_filter: '*APP.app'
    skip_verification: true

- task: ALOpsAppCompiler@1
  displayName: 'ALOPS - Compile Extension: Test'
  inputs:
    usedocker: true
    nav_app_version: '?.?.[yyyyWW].*'
    targetproject: 'LookupValue Extension/Test/app.json'
    app_file_suffix: '_TEST'                                 # A trick to uniquely identify the TEST (from the App-app)

- task: ALOpsAppPublish@1
  displayName: 'ALOPS - Publish Extension'
  inputs:
    usedocker: true
    nav_artifact_app_filter: '*TEST.app'
    skip_verification: true

- task: ALOpsAppTest@1
  displayName: 'ALOPS - Run TestSuite for API'
  inputs:
    usedocker: true
    import_testtoolkit: false
    import_action: 'Skip'
    testpage: '130455'
    testsuite: 'API'
    resultfilename: 'TestResults1.xml'
    failed_test_action: 'Ignore'
  continueOnError: true

- task: ALOpsAppTest@1
  displayName: 'ALOPS - Run TestSuite'
  inputs:
    usedocker: true
    disabledtests: '$(Build.SourcesDirectory)/LookupValue Extension/Test/disabledtests/DisabledTests.json'
    import_testtoolkit: false
    import_action: 'Skip'
    testpage: '130455'
    extensionid: '78ba7129-4003-40b6-8c5b-90d514eb088b'
    resultfilename: 'TestResults2.xml'
    failed_test_action: 'Ignore'
  continueOnError: true

- task: PowerShell@2
  displayName: 'Merge test Results'
  inputs:
    targetType: 'inline'
    script: |
      $TestResults1 = [xml](Get-Content -Path $(Build.SourcesDirectory)/TestResults1.xml)
      $TestResults2 = [xml](Get-Content -Path $(Build.SourcesDirectory)/TestResults2.xml)
      $TestResults2.DocumentElement.ChildNodes | ForEach-Object { $TestResults1.DocumentElement.AppendChild($TestResults1.ImportNode($_, $true)) | Out-Null }
      Set-Content -Path $(Build.SourcesDirectory)/TestResults.xml -Value ($TestResults1.OuterXml)

- task: PublishTestResults@2
  displayName: 'Publish Test Results **/TestResults.xml'
  inputs:
    testResultsFormat: XUnit
    testResultsFiles: '**/TestResults.xml'
    failTaskOnFailedTests: true
    testRunTitle: 'BC Test Results: $(Build.BuildId)'

- task: PowerShell@2
  displayName: "Create DisabledTests.json"
  inputs:
    filePath: '$(Build.SourcesDirectory)/Chapter 13 (LookupValue Extension)/disabledtests/ConvertTo-DisabledTests.ps1'
    arguments: '-inputFolderPath ''$(System.DefaultWorkingDirectory)/'''
    errorActionPreference: 'continue'
  condition: always()

- task: CopyFiles@2
  displayName: 'Copy Test Artifacts to $(Agent.BuildDirectory)/TestResults'
  inputs:
    SourceFolder: '$(Build.Sourcesdirectory)'
    Contents: ?(TestResults.xml|DisabledTests.json)
    TargetFolder: '$(Agent.BuildDirectory)/TestResults'
  condition: always()

- task: PublishPipelineArtifact@1
  displayName: 'Publish Test Artifacts: TestResults.xml and DisabledTests.json'
  inputs:
    targetPath: '$(Agent.BuildDirectory)/TestResults'
    artifact: 'TestResults'
    publishLocation: 'pipeline'
  condition: always()

- task: ALOpsDockerRemove@1
  displayName: 'ALOPS - Remove Docker Container'
  enabled: true
  inputs:
    createsqlbackup: $(createSqlBackup)
  condition: always()

Output

*** Propagated Dependency: Microsoft - Library Variable Storage - 18.0.0.0
*** Propagated Dependency: Microsoft - Tests-TestLibraries - 18.0.0.0
*** Propagated Dependency: Microsoft - Test Runner - 18.0.0.0
*** Download System-Symbols for [18.0.0.0]
*** Downloading NAV Symbols for [System v18.0.0.0]
*** Downloading NAV Symbols from [https://6026cb1916d4:7049/BC/dev/packages?publisher=Microsoft&appName=System&versionText=18.0.0.0&tenant=default]
*** Downloading NAV Symbols with BasicAuthentication [admin]
Invoke-RestMethod -Method Get -Uri https://6026cb1916d4:7049/BC/dev/packages?publisher=Microsoft&appName=System&versionText=18.0.0.0&tenant=default
*** Content-Disposition: attachment; filename=Microsoft_System_22.0.54512.0.app; filename*=UTF-8''Microsoft_System_22.0.54512.0.app
*** Saving symbol file [Microsoft_System_22.0.54512.0.app] to [c:\Run\DevOps\PackageCache]
*** Read App Manifest: [c:\Run\DevOps\PackageCache\Microsoft_System_22.0.54512.0.app].
***
*** Download Application-Symbols for [18.0.0.0]
*** Downloading NAV Symbols for [Application v18.0.0.0]
*** Downloading NAV Symbols from [https://6026cb1916d4:7049/BC/dev/packages?publisher=Microsoft&appName=Application&versionText=18.0.0.0&tenant=default]
*** Downloading NAV Symbols with BasicAuthentication [admin]
Invoke-RestMethod -Method Get -Uri https://6026cb1916d4:7049/BC/dev/packages?publisher=Microsoft&appName=Application&versionText=18.0.0.0&tenant=default
*** Content-Disposition: attachment; filename=Microsoft_Application_22.0.54157.54635.app; filename*=UTF-8''Microsoft_Application_22.0.54157.54635.app
*** Saving symbol file [Microsoft_Application_22.0.54157.54635.app] to [c:\Run\DevOps\PackageCache]
*** Read App Manifest: [c:\Run\DevOps\PackageCache\Microsoft_Application_22.0.54157.54635.app].
*** Propagated Dependency: Microsoft - System Application - 22.0.0.0
*** Propagated Dependency: Microsoft - Base Application - 22.0.0.0
***
*** Check for Depency Apps 8
*** Download Application-Symbols for [LookupValue v2.0.0.0]
*** Downloading NAV Symbols for [LookupValue v2.0.0.0]
*** Downloading NAV Symbols from [https://6026cb1916d4:7049/BC/dev/packages?publisher=fluxxus.nl&appName=LookupValue&versionText=2.0.0.0&tenant=default&appid=e26890f8-fafe-49c6-8951-2c1457921f9b]
*** Downloading NAV Symbols with BasicAuthentication [admin]
Invoke-RestMethod -Method Get -Uri https://6026cb1916d4:7049/BC/dev/packages?publisher=fluxxus.nl&appName=LookupValue&versionText=2.0.0.0&tenant=default&appid=e26890f8-fafe-49c6-8951-2c1457921f9b
*** Content-Disposition: attachment; filename=fluxxus.nl_LookupValue_2.0.202314.1491.app; filename*=UTF-8''fluxxus.nl_LookupValue_2.0.202314.1491.app
*** Saving symbol file [fluxxus.nl_LookupValue_2.0.202314.1491.app] to [c:\Run\DevOps\PackageCache]
*** Read App Manifest: [c:\Run\DevOps\PackageCache\fluxxus.nl_LookupValue_2.0.202314.1491.app].
***
*** Download Application-Symbols for [Any v18.0.0.0]
*** Downloading NAV Symbols for [Any v18.0.0.0]
*** Downloading NAV Symbols from [https://6026cb1916d4:7049/BC/dev/packages?publisher=Microsoft&appName=Any&versionText=18.0.0.0&tenant=default&appid=e7320ebb-08b3-4406-b1ec-b4927d3e280b]
*** Downloading NAV Symbols with BasicAuthentication [admin]
Invoke-RestMethod -Method Get -Uri https://6026cb1916d4:7049/BC/dev/packages?publisher=Microsoft&appName=Any&versionText=18.0.0.0&tenant=default&appid=e7320ebb-08b3-4406-b1ec-b4927d3e280b
##[error]Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (404) Not Found."
Async Command Start: Add Build Tag
Build '1491' has following tags now: AL.Compiler.ALLanguage, BC.Platform.22.0.54512.0, myimage-onprem-22.0.54157.54635-w1-10.0.20348.587-ltsc2022, BC.Version.22.0.54157.54635, App.LookupValue.2.0.202314.1491
Async Command End: Add Build Tag
Async Command Start: Update Build Number
Update build number to 2.0.202314.1491 for build 1491
Async Command End: Update Build Number
Finishing: ALOPS - Compile Extension: Test

Expected behavior

Test library apps have been installed

AdminHodor commented 1 year ago

Dear @lvanvugt ,

This seems like a problem with the images. Please note Microsoft made some changes to the generic-images for v22, but.... they only updated the generic-images for SUPPORTED Operating systems. Hence, if you are running an outdated OS, the "Get-BestGenericImageForOs" will return an image that is not updated for v22. The code searches for a specific PSD1 file, which is moved for v22, if it's not found the container will skip the entire "Install Test-Toolkit" step.

2 solutions for this:

update your servers/hosts so you can use the updated generic images.
manipulate the images to include the updates files for v22

Kind regards,

AdminHodor commented 1 year ago

Dear @lvanvugt,

If you want to manipulate the files in the images, here is the source: https://github.com/microsoft/nav-docker/tree/master/generic/Run

It are the files you can find in "210-new" folder that contain the updates for BC v22.

Kind regards,

lvanvugt commented 1 year ago

Thanx!

Hence, if you are running an outdated OS, the "Get-BestGenericImageForOs" will return an image that is not updated for v22.

Do you know where I can find which OS versions are outdated?

lvanvugt commented 1 year ago

I have solved the issue by running Windows updates on my agents server. Something which is not done automatically. Was considering to create new Azure VMs, but thought I could give it a try with Windows update. So far, so good.

The suggested option seemed a bit to tedious in all honesty.

HodorNV / ALOps