AL-Pipeline: Table "XXXX" for the extension object is not found, despite symbols being installed correctly

[EsatSayinkaplanAccelent]

Describe the issue Pipeline is set up in DevOps with 3rd Party Apps and Dependencies located in InstallApps Folder. The sync and publish step of the 3rd parties works without any errors. The problem is later on when the app folders are gettings compiled with external dependencies. Some Apps do compile correctly with packages but one of the big ones throws the error: The target Table 'XXXX' for the extension object is not found, despite the apps definitely being already installed. The symbols used are either directly provided by the partners or downloaded from the production env. via "download symbols". If compiled via VS Code, it works.

Dont mind the override to input the DLLs.

If you need additional Info, feel free to ask.

Scripts used to create container and cause the issue

Run-AlPipeline @params `
    -pipelinename $pipelineName `
    -containerName $containerName `
    -imageName $imageName `
    -bcAuthContext $authContext `
    -environment $environmentName `
    -artifact $artifact.replace('{INSIDERSASTOKEN}',$insiderSasToken) `
    -memoryLimit $memoryLimit `
    -baseFolder $baseFolder `
    -licenseFile $LicenseFile `
    -installApps $installApps `
    -installTestApps $installTestApps `
    -previousApps $previousApps `
    -appFolders $appFolders `
    -testFolders $testFolders `
    -doNotRunTests:$doNotRunTests `
    -testResultsFile $testResultsFile `
    -testResultsFormat 'JUnit' `
    -installTestRunner:$installTestRunner `
    -installTestFramework:$installTestFramework `
    -installTestLibraries:$installTestLibraries `
    -installPerformanceToolkit:$installPerformanceToolkit `
    -enableCodeCop:$enableCodeCop `
    -enableAppSourceCop:$enableAppSourceCop `
    -enablePerTenantExtensionCop:$enablePerTenantExtensionCop `
    -enableUICop:$enableUICop `
    -azureDevOps:($environment -eq 'AzureDevOps') `
    -gitLab:($environment -eq 'GitLab') `
    -gitHubActions:($environment -eq 'GitHubActions') `
    -failOn 'error' `
    -AppSourceCopMandatoryAffixes $appSourceCopMandatoryAffixes `
    -AppSourceCopSupportedCountries $appSourceCopSupportedCountries `
    -additionalCountries $additionalCountries `
    -buildArtifactFolder $buildArtifactFolder `
    -CreateRuntimePackages:$CreateRuntimePackages `
    -appBuild $appBuild -appRevision $appRevision `
    -credential $credential `
    -NewBcContainer {
        Param([Hashtable]$parameters)
        New-BcContainer @parameters

        $dllFiles = Get-ChildItem -Path '.\.netpackages\' -Recurse -Filter '*.dll' -File -Force -Verbose
        foreach ($dll in $dllFiles) {
            Copy-FileToBcContainer -containerName $containerName -localPath $dll.FullName -containerPath "C:\Program Files\Microsoft Dynamics NAV\230\Service\Add-Ins\$($dll.Name)"
            # Copy-FileToBcContainer -containerName $containerName -localPath $dll.FullName -containerPath "C:\Program Files\dotnet\shared\$($dll.Name)"
        }

        $serverdllFiles = Get-ChildItem -Path '.\.resources\ServerAddIns\' -Recurse -Filter '*.dll' -File -Force -Verbose
        foreach ($serverdll in $serverdllFiles) {
            Copy-FileToBcContainer -containerName $containerName -localPath $serverdll.FullName -containerPath "C:\Program Files\Microsoft Dynamics NAV\230\Service\Add-Ins\$($serverdll.Name)"
            # Copy-FileToBcContainer -containerName $containerName -localPath $serverdll.FullName -containerPath "C:\Program Files\dotnet\shared\$($serverdll.Name)"
        }

        Invoke-ScriptInBcContainer $parameters.ContainerName -scriptblock {
            $progressPreference = 'SilentlyContinue'
            Set-NavServerInstance -ServerInstance 'BC' -Restart
        }
    } `
    -PublishBcContainerApp {
        Param([HashTable] $parameters)
        $parameters.sync = $true 
        $parameters.syncMode = 'ForceSync'
        Publish-BcContainerApp @parameters
    }

Full output of scripts

Processing dependency CKL Software GmbH_Kostenrechnung 365 / Cost Accounting 365_6.1.13.0 (32a8e514-216f-487a-82ea-23709b370af9)
Downloading symbols: CKL Software GmbH_Kostenrechnung 365  Cost Accounting 365_6.1.17.0.app
Url : .../packages?appId=32a8e514-216f-487a-82ea-23709b370af9&versionText=6.1.13.0&tenant=default

Compiling...
.\alc.exe /project:"c:\sources\XXXX" /packagecachepath:"c:\sources\.packages" /out:"c:\sources\.output\XXXXX.Base_1.0.11639.0.app" /BuildBy:"BcContainerHelper,6.0.26-preview1247" /assemblyprobingpaths:"C:\Program Files\dotnet\shared","C:\ProgramData\BcContainerHelper\Extensions\XXXXX\.netPackages\Service"
Microsoft (R) AL Compiler version 12.7.14.31432
Copyright (C) Microsoft Corporation. All rights reserved

Compilation started for project 'XXX.Base' containing '1604' files at '12:37:42.793'.

##[error]./c:/sources/XXXX/src/tableextension/BaseAllocationAssignmentCOA.TableExt.al(1,63): error AL0247: The target Table 'Allocation Assignment COA' for the extension object is not found
##[error]./c:/sources/XXXX/src/tableextension/BaseCostAccJnlLine.TableExt.al(1,57): error AL0247: The target Table 'Cost Acc. Journal Line COA' for the extension object is not found
##[error]./c:/sources/XXXX/src/tableextension/BaseCostAccLedgEntry.TableExt.al(1,59): error AL0247: The target Table 'Cost Acc. Ledger Entry COA' for the extension object is not found

[freddydk]

You would need to include the full script.

[EsatSayinkaplanAccelent]

Ist mostly based on the old ms repo for devops.

Param(
    [Parameter(Mandatory=$false)]
    [ValidateSet('AzureDevOps','GithubActions','GitLab')]
    [string] $environment = 'AzureDevOps',
    [Parameter(Mandatory=$true)]
    [string] $version,
    [Parameter(Mandatory=$false)]
    [int] $appBuild = 0,
    [Parameter(Mandatory=$false)]
    [int] $appRevision = 0
)

if ($environment -eq "AzureDevOps") {
    $buildArtifactFolder = $ENV:BUILD_ARTIFACTSTAGINGDIRECTORY
}
elseif ($environment -eq "GitHubActions") {
    $buildArtifactFolder = Join-Path $ENV:GITHUB_WORKSPACE "output"
    New-Item $buildArtifactFolder -ItemType Directory | Out-Null
}

# XXX-DOC: Determine base Folder from PSScriptRoot
$scriptFullPath = (Get-Item (Join-Path $PSScriptRoot "...")).FullName
$baseFolder = (Get-Item (Join-Path $scriptFullPath "..\..")).FullName
. (Join-Path $PSScriptRoot "Read-Settings.ps1") -environment $environment -version $version
. (Join-Path $PSScriptRoot "Install-BcContainerHelper.ps1") -bcContainerHelperVersion $bcContainerHelperVersion -genericImageName $genericImageName

Write-Host "Using BaseFolder: $baseFolder"

$authContext = $null
$refreshToken = "$($ENV:BcSaasRefreshToken)"
$environmentName = "$($ENV:EnvironmentName)"
if ($refreshToken -and $environmentName) {
    $authContext = New-BcAuthContext -refreshToken $refreshToken
    if (Get-BcEnvironments -bcAuthContext $authContext | Where-Object { $_.Name -eq $environmentName -and  $_.type -eq "Sandbox" }) {
        Remove-BcEnvironment -bcAuthContext $authContext -environment $environmentName
    }
    $countryCode = $artifact.Split('/')[3]
    New-BcEnvironment -bcAuthContext $authContext -environment $environmentName -countryCode $countrycode -environmentType "Sandbox" | Out-Null
    do {
        Start-Sleep -Seconds 10
        $baseApp = Get-BcPublishedApps -bcAuthContext $authContext -environment $environmentName | Where-Object { $_.Name -eq "Base Application" }
    } while (!($baseApp))
    $baseapp | Out-Host

    $artifact = Get-BCArtifactUrl `
        -country $countryCode `
        -version $baseApp.Version `
        -select Closest

    if ($artifact) {
        Write-Host "Using Artifacts: $artifact"
    }
    else {
        throw "No artifacts available"
    }
}

$params = @{}
$insiderSasToken = "$ENV:insiderSasToken"
$licenseFile = "$ENV:licenseFile"
$codeSigncertPfxFile = "$ENV:CodeSignCertPfxFile"
if (!$doNotSignApps -and $codeSigncertPfxFile) {
    if ("$ENV:CodeSignCertPfxPassword" -ne "") {
        $codeSignCertPfxPassword = try { "$ENV:CodeSignCertPfxPassword" | ConvertTo-SecureString } catch { ConvertTo-SecureString -String "$ENV:CodeSignCertPfxPassword" -AsPlainText -Force }
        $params = @{
            "codeSignCertPfxFile" = $codeSignCertPfxFile
            "codeSignCertPfxPassword" = $codeSignCertPfxPassword
        }
    }
    else {
        $codeSignCertPfxPassword = $null
    }
}

$allTestResults = "testresults*.xml"
$testResultsFile = Join-Path $baseFolder "TestResults.xml"
$testResultsFiles = Join-Path $baseFolder $allTestResults
if (Test-Path $testResultsFiles) {
    Remove-Item $testResultsFiles -Force
}

# XXX-DOC: Set credentials for persistent use, adviced to be used with -keepContainers and reUseContainer
$credential = New-Object pscredential 'admin', (ConvertTo-SecureString -String 'admin' -AsPlainText -Force)

# XXX-DOC: Sets the AppFolders to compile
$appFolders = @(Get-ChildItem -Path ".\XXX*" -Directory)

# XXX-DOC: AL Language to use, gets loaded from Artifact
$vsixFile = Get-AlLanguageExtensionFromArtifacts -artifactUrl (Get-BCArtifactUrl -type OnPrem -country "de" -version "23.5." -select Latest) -Verbose
Write-Host "Using vsix-File located in $vsixFile"

Run-AlPipeline @params `
    -pipelinename $pipelineName `
    -containerName $containerName `
    -imageName $imageName `
    -bcAuthContext $authContext `
    -environment $environmentName `
    -artifact $artifact.replace('{INSIDERSASTOKEN}',$insiderSasToken) `
    -memoryLimit $memoryLimit `
    -baseFolder $baseFolder `
    -licenseFile $LicenseFile `
    -installApps $installApps `
    -installTestApps $installTestApps `
    -previousApps $previousApps `
    -appFolders $appFolders `
    -testFolders $testFolders `
    -doNotRunTests:$doNotRunTests `
    -testResultsFile $testResultsFile `
    -testResultsFormat 'JUnit' `
    -installTestRunner:$installTestRunner `
    -installTestFramework:$installTestFramework `
    -installTestLibraries:$installTestLibraries `
    -installPerformanceToolkit:$installPerformanceToolkit `
    -enableCodeCop:$enableCodeCop `
    -enableAppSourceCop:$enableAppSourceCop `
    -enablePerTenantExtensionCop:$enablePerTenantExtensionCop `
    -enableUICop:$enableUICop `
    -azureDevOps:($environment -eq 'AzureDevOps') `
    -gitLab:($environment -eq 'GitLab') `
    -gitHubActions:($environment -eq 'GitHubActions') `
    -failOn 'error' `
    -AppSourceCopMandatoryAffixes $appSourceCopMandatoryAffixes `
    -AppSourceCopSupportedCountries $appSourceCopSupportedCountries `
    -additionalCountries $additionalCountries `
    -buildArtifactFolder $buildArtifactFolder `
    -CreateRuntimePackages:$CreateRuntimePackages `
    -appBuild $appBuild -appRevision $appRevision `
    -credential $credential `
    -NewBcContainer {
        Param([Hashtable]$parameters)
        New-BcContainer @parameters

        $dllFiles = Get-ChildItem -Path '.\.netpackages\' -Recurse -Filter '*.dll' -File -Force -Verbose
        foreach ($dll in $dllFiles) {
            Copy-FileToBcContainer -containerName $containerName -localPath $dll.FullName -containerPath "C:\Program Files\Microsoft Dynamics NAV\230\Service\Add-Ins\$($dll.Name)"
            # Copy-FileToBcContainer -containerName $containerName -localPath $dll.FullName -containerPath "C:\Program Files\dotnet\shared\$($dll.Name)"
        }

        $serverdllFiles = Get-ChildItem -Path '.\.resources\ServerAddIns\' -Recurse -Filter '*.dll' -File -Force -Verbose
        foreach ($serverdll in $serverdllFiles) {
            Copy-FileToBcContainer -containerName $containerName -localPath $serverdll.FullName -containerPath "C:\Program Files\Microsoft Dynamics NAV\230\Service\Add-Ins\$($serverdll.Name)"
            # Copy-FileToBcContainer -containerName $containerName -localPath $serverdll.FullName -containerPath "C:\Program Files\dotnet\shared\$($serverdll.Name)"
        }

        Invoke-ScriptInBcContainer $parameters.ContainerName -scriptblock {
            $progressPreference = 'SilentlyContinue'
            Set-NavServerInstance -ServerInstance 'BC' -Restart
        }
    } `
    -PublishBcContainerApp {
        Param([HashTable] $parameters)
        $parameters.sync = $true 
        $parameters.syncMode = 'ForceSync'
        Publish-BcContainerApp @parameters
    }

if ($environment -eq 'AzureDevOps') {
    Write-Host "##vso[task.setvariable variable=TestResults]$allTestResults"
}

[EsatSayinkaplanAccelent]

@freddydk inserted full script

[freddydk]

and the full output?

[EsatSayinkaplanAccelent]

Too long for GitHub so its in the txt file Error Full Output.txt @freddydk

[EsatSayinkaplanAccelent]

@freddydk quick additional info from not directly related issue #3604 To confirm that the used package is a valid one I ran Extract-AppFileToFolder with it, to check its integrity. The extraction worked and in the extracted SymbolReference.json there are definitely the hints to the missing table "Allocation Assignment COA". Hope this helps.

[freddydk]

All the external apps you have - are they runtime packages, full apps or symbol apps?

[EsatSayinkaplanAccelent]

If possible I used the runtime versions. The first error throwing app "CKL Kostenrechnung 365 / Cost Accounting 365" is an symbol app now after the full app did not work. The error further down with "ECM" in it is from EASY SOFTWARE AG and as runtime version.

[freddydk]

The way it works (by default) is to publish all apps from installedApps to the container and then when building apps - downloading the symbols from the container for the compiler. This process doesn't work if you use symbol apps. If you add -doNotPublishApps to your Run-AlPipeline, then it might build - but you won't be able to run tests (but you cannot do that anyway if some of your apps are symbol apps)

[EsatSayinkaplanAccelent]

Alright, as far as I can see the customer does not use tests. I will the try the -doNotPublishApps switch. Is there a way to check if an app is a symbol, full or runtime app?

And ofc thx for your input.

[EsatSayinkaplanAccelent]

After looking into https://github.com/microsoft/navcontainerhelper/blob/main/AppHandling/Run-AlPipeline.ps1 for the -doNotPublishApps, does this mean I would need to use $useCompilerFolder? And do I need a database for the pipeline to compile against, or does that work with just the compiler?

[freddydk]

You can use compilerfolder if you want - else it will use a filesonly container.

[EsatSayinkaplanAccelent]

I think I have to use it, else the .NET-DLLs are not there for compilation? Or do I have a thinking error rn?

[EsatSayinkaplanAccelent]

@freddydk quick update, thanks to your help it worked with a filesonly container. But somehow the script does not get to seem the $buildArtifactFolder = $ENV:BUILD_ARTIFACTSTAGINGDIRECTORY right.

I get the following output after all extensions get compiled:

Compilation ended at '13:05:36.328'.

C:\azure_pipeline\bcnas01_agent1\_work\1\s\.output\XXX.Corp.Support_1.0.11666.0.app successfully created in 88 seconds
XXX.Corp.Support_1.0.11666.0.app copied to C:\azure_pipeline\bcnas01_agent1\_work\1\s\.packages

Compiling apps took 1720 seconds
Copy to build artifacts
  _____                     _          _           _ _     _              _   _  __           _
 / ____|                   | |        | |         (_) |   | |            | | (_)/ _|         | |
 | |     ___  _ __  _   _  | |_ ___   | |__  _   _ _| | __| |   __ _ _ __| |_ _| |_ __ _  ___| |_ ___
 | |    / _ \| '_ \| | | | | __/ _ \  | '_ \| | | | | |/ _` |  / _` | '__| __| |  _/ _` |/ __| __/ __|
 | |___| (_) | |_) | |_| | | || (_) | | |_) | |_| | | | (_| | | (_| | |  | |_| | || (_| | (__| |_\__ \
 \______\___/| .__/ \__, |  \__\___/  |_.__/ \__,_|_|_|\__,_|  \__,_|_|   \__|_|_| \__,_|\___|\__|___/
             | |     __/ |
             |_|    |___/
Files in build artifacts folder:

Copying to Build Artifacts took 0 seconds

Any Idea?

[freddydk]

In your log, the buildartifactsFolder seems to be the same as the basefolder:

2024-10-10T10:52:09.8288772Z BuildArtifactFolder C:\azure_pipeline\bcnas01_agent1_work\1\a

(or maybe it is empty - don't know)

In AL-Go I use this parameter to specify where I want the artifacts.

[freddydk]

I think I have to use it, else the .NET-DLLs are not there for compilation? Or do I have a thinking error rn?

A Files-Only container will have the DLLs as well

[freddydk]

Looks like the build now succeeds, will close this issue.