Running tests fails with 401 when running on a BC25 sandbox container with AAD authentication

[amea20]

Describe the issue When trying to run tests in the current sandbox container with country w1 and AAD authentication, it fails with a 401.

Scripts used to create container and cause the issue

$artifactUrl = Get-BCArtifactUrl -type Sandbox -select Current -country w1
$userName = "bcadmin"
$password = ConvertTo-SecureString -String "Passwordhere" -AsPlainText -Force
$credential = New-Object -TypeName "System.Management.Automation.PSCredential" -ArgumentList $userName, $password
$licenseFileUri = 'C:\Licenses\BC-License.bclicense'
$additionalParameters = @("--volume ""C:\Apps:C:\Agents""")
$aadUser = '' # EntraID user name
$aadPassword = '' # EntraID password
$aadcredential = ([PSCredential]::new($aadUser, (ConvertTo-SecureString -String $aadPassword -AsPlainText -Force)))
$bcAuthcontext = New-BcAuthContext -credential $aadcredential -tenantID $tenantId

New-BCContainer -accept_eula `
                -Credential $credential `
                -licenseFile $licenseFileUri `
                -artifactUrl $artifactUrl `
                -containerName bc `
                -auth AAD `
                -doNotCheckHealth `
                -memoryLimit 20G `
                -AadTenant $tenantId `
                -AadAppId $appId `
                -AadAppIdUri $appIdUri `
                -authenticationEMail $userPrincipalEmail `
                -updateHosts `
                -multitenant:$false `
                -includeTestToolkit `
                -includeTestLibrariesOnly `
                -additionalParameters $additionalParameters `
                -useBestContainerOS

Run-TestsInBCContainer -containerName bc -credential $credential -accessToken $bcAuthcontext.AccessToken -testSuite 'DEFAULT' 

Full output of scripts

BcContainerHelper version 6.0.24
BC.HelperFunctions emits usage statistics telemetry to Microsoft
Running on Windows, PowerShell 5.1.19041.4894
Using Container
Connecting to http://localhost:80/BC/cs?tenant=default
CommunicationError : Response status code does not indicate success: 401 (Unauthorized).
at <ScriptBlock>, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 93
at AwaitState, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 222
at OpenSession, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 200
at Initialize, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 61
at ClientContext, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 20
at New-ClientContext, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\PsTestFunctions.ps1: line 48
at <ScriptBlock>, <No file>: line 58
at AwaitState, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 234
at OpenSession, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 200
at Initialize, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 61
at ClientContext, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 20
at New-ClientContext, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\PsTestFunctions.ps1: line 48
at <ScriptBlock>, <No file>: line 58
ClientSession State is Uninitialized (Wait time 10 seconds)

Exception Script Stack Trace:
at AwaitState, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 234
at OpenSession, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 200
at Initialize, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 61
at ClientContext, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\ClientContext.ps1: line 20
at New-ClientContext, C:\ProgramData\BcContainerHelper\Extensions\bc\PsTestTool\PsTestFunctions.ps1: line 48
at <ScriptBlock>, <No file>: line 58

PowerShell Call Stack:
at Invoke-ScriptInBcContainer, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.24\ContainerHandling\Invoke-ScriptInNavContainer.ps1: line 71
at Run-TestsInBcContainer, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.24\AppHandling\Run-TestsInNavContainer.ps1: line 430
at <ScriptBlock>, <No file>: line 1

Container Free Physical Memory: 17.1Gb
Disk C: Free 122Gb from 127Gb

Services in container bc:
- MicrosoftDynamicsNavServer$BC is Running
- MSSQL$SQLEXPRESS is Running

Relevant event log from container bc:
- 20241003 12:50:20 - MicrosoftDynamicsNAVClientWebClient

  Category: Microsoft.IdentityModel.S2S.Extensions.AspNetCore.S2SAuthenticationHandler
  EventId: 0
  ConnectionId: 0HN73HE3S4D58
  RequestId: 0HN73HE3S4D58:00000001
  RequestPath: /BC/cs

  CorrelationId: 330e15c3-8e57-4c70-a767-0e22b0d10335
  Exception: Microsoft.IdentityModel.Tokens.SecurityTokenInvalidAudienceException: IDX10214: Audience validation failed. Audiences: 'https://api.businesscentral.dynamics.com'. Did 
not match: validationParameters.ValidAudience: 'null' or validationParameters.ValidAudiences: 'null, dd395f20-2cf2-4e15-a4e3-2bf88691b000, null, dd395f20-2cf2-4e15-a4e3-2bf88691b000'.
     at Microsoft.IdentityModel.Tokens.Validators.ValidateAudience(IEnumerable`1 audiences, SecurityToken securityToken, TokenValidationParameters validationParameters)
     at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateTokenPayloadAsync(JsonWebToken jsonWebToken, TokenValidationParameters validationParameters, BaseConfigura
tion configuration)
     at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateJWSAsync(JsonWebToken jsonWebToken, TokenValidationParameters validationParameters, BaseConfiguration conf
iguration)
  Detailed stack trace is only included at verbose log level! To log full stack trace see https://aka.ms/sal/stacktrace-help.

- 20241003 12:50:20 - MicrosoftDynamicsNAVClientWebClient

  Category: Microsoft.IdentityModel.S2S.Extensions.AspNetCore.S2SAuthenticationHandler
  EventId: 0
  ConnectionId: 0HN73HE3S4D58
  RequestId: 0HN73HE3S4D58:00000001
  RequestPath: /BC/cs

  IDX10214: Audience validation failed. Audiences: 'https://api.businesscentral.dynamics.com'. Did not match: validationParameters.ValidAudience: 'null' or validationParameters.Val
idAudiences: 'null, dd395f20-2cf2-4e15-a4e3-2bf88691b000, null, dd395f20-2cf2-4e15-a4e3-2bf88691b000'.

at Invoke-ScriptInBcContainer, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.24\ContainerHandling\Invoke-ScriptInNavContainer.ps1: line 113
at Run-TestsInBcContainer, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.24\AppHandling\Run-TestsInNavContainer.ps1: line 430
at <ScriptBlock>, <No file>: line 1
Run-TestsInBCContainer Telemetry Correlation Id: dc6c6e08-5e2c-4c45-b848-caad5197a581
ClientSession State is Uninitialized (Wait time 10 seconds)
At C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.24\AppHandling\Run-TestsInNavContainer.ps1:570 char:17
+                 throw $_.Exception.Message
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (ClientSession S...ime 10 seconds):String) [], RuntimeException
    + FullyQualifiedErrorId : ClientSession State is Uninitialized (Wait time 10 seconds)

Screenshots none

Additional context This happens all the time with the BC25 Sandbox container It used to work 2 days ago when the current version was BC24.5

[freddydk]

Could you supply the container generation log?

[freddydk]

Also - can you login to the WebClient using AAD Auth?

[amea20]

Hi Freddy,

There were no issues compiling and publishing apps or logging in to the web client.

Here's the container generation log:

BcContainerHelper is version 6.0.24
BcContainerHelper is running as administrator
HyperV is Enabled
Host is Microsoft Windows 10 Pro - 10.0.19045.4894
UsePsSession is True
UsePwshForBc24 is True
UseWinRmSession is allow
UseSslForWinRmSession is True
Docker Client Version is 23.0.1
Docker Server Version is 23.0.1
Removing Session bc
Removing container bc
Removing entries from hosts
Removing bc from container hosts file
Removing bc-* from container hosts file
Removing Desktop shortcuts
Removing C:\ProgramData\BcContainerHelper\Extensions\bc
Fetching all docker images
Fetching all docker volumes
Using image mcr.microsoft.com/businesscentral:ltsc2019
Disabling Health Check (always report healthy)
Creating Container bc
Style: sandbox
Multitenant: No
Version: 25.0.23364.25044
Platform: 25.0.24985.0
Generic Tag: 1.0.2.39
Container OS Version: 10.0.17763.6054 (ltsc2019)
Host OS Version: 10.0.19045.4894 (22H2)
Using hyperv isolation
Using locale en-US
Disabling the standard eventlog dump to container log every 2 seconds (use -dumpEventLog to enable)
Using license file C:\NavOne\Licenses\BC24-License.bclicense
Additional Parameters:
--volume "C:\work\NavOne:C:\Agents"
--expose 5986
--env customNavSettings=ValidAudiences=dd395f20-2cf2-4e15-a4e3-2bf88691b000;https://api.businesscentral.dynamics.com,DisableTokenSigningCertificateValidation=True,ExtendedSecurityT
okenLifetime=24,ClientServicesCredentialType=NavUserPassword
--env customWebSettings=AadApplicationId=dd395f20-2cf2-4e15-a4e3-2bf88691b000,AadAuthorityUri=https://login.microsoftonline.com/domain.com
Files in C:\ProgramData\BcContainerHelper\Extensions\bc\my:
- AdditionalOutput.ps1
- AdditionalSetup.ps1
- HelperFunctions.ps1
- license.bclicense
- MainLoop.ps1
- SetupVariables.ps1
- updatehosts.ps1
Creating container bc from image mcr.microsoft.com/businesscentral:ltsc2019
b5d188c1b4f05316867dc028cd0d30e858b5b0fafe2435b4c4e66789a88a8f2a
Waiting for container bc to be ready
Using artifactUrl https://bcartifacts-exdbf9fwegejdqak.b02.azurefd.net/sandbox/25.0.23364.25044/w1
Using installer from C:\Run\240
Installing Business Central: multitenant=False, installOnly=False, filesOnly=False, includeTestToolkit=False, includeTestLibrariesOnly=False, includeTestFrameworkOnly=False, includ
ePerformanceToolkit=False, appArtifactPath=c:\dl\sandbox\25.0.23364.25044\w1, platformArtifactPath=c:\dl\sandbox\25.0.23364.25044\platform, databasePath=c:\dl\sandbox\25.0.23364.25
044\w1\BusinessCentral-W1.bak, licenseFilePath=c:\dl\sandbox\25.0.23364.25044\w1\Cronus.bclicense, rebootContainer=True
Installing from artifacts
Starting Local SQL Server
Starting Internet Information Server
Copying Service Tier Files
c:\dl\sandbox\25.0.23364.25044\platform\ServiceTier\Program Files
c:\dl\sandbox\25.0.23364.25044\platform\ServiceTier\System64Folder
Copying Web Client Files
c:\dl\sandbox\25.0.23364.25044\platform\WebClient\Microsoft Dynamics NAV
Copying ModernDev Files
c:\dl\sandbox\25.0.23364.25044\platform
c:\dl\sandbox\25.0.23364.25044\platform\ModernDev\program files\Microsoft Dynamics NAV
Copying additional files
Copying ConfigurationPackages
C:\dl\sandbox\25.0.23364.25044\platform\ConfigurationPackages
Copying Test Assemblies
C:\dl\sandbox\25.0.23364.25044\platform\Test Assemblies
Copying Extensions
C:\dl\sandbox\25.0.23364.25044\w1\Extensions
Copying Applications
C:\dl\sandbox\25.0.23364.25044\platform\Applications
Copying dependencies
Importing PowerShell Modules
Restoring CRONUS Demo Database
Setting CompatibilityLevel for CRONUS on localhost\SQLEXPRESS
Modifying Business Central Service Tier Config File for Docker
Creating Business Central Service Tier
Installing SIP crypto provider: 'C:\Windows\System32\NavSip.dll'
Starting Business Central Service Tier
Importing license file
Stopping Business Central Service Tier
Installation took 116 seconds
Installation complete
Initializing...
Setting kubernetes.docker.internal to 127.0.0.1 in container hosts file (copy from host hosts file)
Setting host.docker.internal to 192.168.1.189 in container hosts file (copy from host hosts file)
Setting host.containerhelper.internal to 172.23.96.1 in container hosts file
Starting Container
Hostname is bc
PublicDnsName is bc
Using AccessControlService Authentication
Creating Self Signed Certificate
Self Signed Certificate Thumbprint 216D170A742D452E73BD2DFC495744C239090165
DNS identity bc
Modifying Service Tier Config File with Instance Specific Settings
Modifying Service Tier Config File with settings from environment variable
Setting ValidAudiences to dd395f20-2cf2-4e15-a4e3-2bf88691b000;https://api.businesscentral.dynamics.com
Setting DisableTokenSigningCertificateValidation to True
Setting ExtendedSecurityTokenLifetime to 24
Setting ClientServicesCredentialType to NavUserPassword
Starting Service Tier
Registering event sources
Creating DotNetCore Web Server Instance
Using application pool name: BC
Using default container name: NavWebApplicationContainer
Copy files to WWW root C:\inetpub\wwwroot\BC
Create the application pool BC
Create website: NavWebApplicationContainer without SSL
Update configuration: navsettings.json
Done Configuring Web Client
Modifying Web Client config with settings from environment variable
Setting AadApplicationId to dd395f20-2cf2-4e15-a4e3-2bf88691b000
Setting AadAuthorityUri to https://login.microsoftonline.com/domain.com
Enabling Financials User Experience
Using license file 'c:\run\my\license.bclicense'
Import License
Creating http download site
Setting SA Password and enabling SA
Creating bcadmin as SQL User and add to sysadmin
WARNING: This license is not compatible with this version of Business Central.
Creating SUPER user
WARNING: This license is not compatible with this version of Business Central.
WARNING: This license is not compatible with this version of Business Central.
Enable PSRemoting and setup user for winrm
Creating self-signed certificate for winrm
Container IP Address: 172.23.102.209
Container Hostname  : bc
Container Dns Name  : bc
Web Client          : http://bc/BC/
Dev. Server         : http://bc
Dev. ServerInstance : BC
Setting bc to 172.23.102.209 in host hosts file

Files:
http://bc:8080/ALLanguage.vsix

WARNING: You are running a container which is 73 days old.
Microsoft recommends that you always run the latest version of our containers.

Container Total Physical Memory is 20.5Gb
Container Free Physical Memory is 17.4Gb

Initialization took 46 seconds
Ready for connections!
Reading CustomSettings.config from bc
Creating Desktop Shortcuts for bc
Synchronizing Permissions Mock on default
App successfully synchronized
Installing Permissions Mock on default
App successfully installed
Synchronizing Test Runner on default
App successfully synchronized
Installing Test Runner on default
App successfully installed
Synchronizing Any on default
App successfully synchronized
Installing Any on default
App successfully installed
Synchronizing Library Assert on default
App successfully synchronized
Installing Library Assert on default
App successfully installed
Skipping app 'C:\Applications\TestFramework\TestLibraries\permissions mock\Microsoft_Permissions Mock.app' as it is already installed
Synchronizing Library Variable Storage on default
App successfully synchronized
Installing Library Variable Storage on default
App successfully installed
Publishing C:\ProgramData\BcContainerHelper\Extensions\bc\669c031d-e58e-4944-8dac-651488d58da8\Microsoft_System Application Test Library.app
Synchronizing System Application Test Library on tenant default
Installing System Application Test Library on tenant default
App Microsoft_System Application Test Library.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\bc\ab36cd6b-fe04-4ea0-9f7e-3ed22a5de63c\Microsoft_Business Foundation Test Libraries.app
Synchronizing Business Foundation Test Libraries on tenant default
Installing Business Foundation Test Libraries on tenant default
App Microsoft_Business Foundation Test Libraries.app successfully published
Publishing C:\ProgramData\BcContainerHelper\Extensions\bc\96d2024d-4684-4e0f-ae9f-a8be35fc2958\Microsoft_Tests-TestLibraries.app
Synchronizing Tests-TestLibraries on tenant default
Installing Tests-TestLibraries on tenant default
App Microsoft_Tests-TestLibraries.app successfully published
Synchronizing AI Test Toolkit on default
App successfully synchronized
Installing AI Test Toolkit on default
App successfully installed
TestToolkit successfully imported
Cleanup old dotnet core assemblies
Container bc successfully created

Use:
Get-BcContainerEventLog -containerName bc to retrieve a snapshot of the event log from the container
Get-BcContainerDebugInfo -containerName bc to get debug information about the container
Enter-BcContainer -containerName bc to open a PowerShell prompt inside the container
Remove-BcContainer -containerName bc to remove the container again
docker logs bc to retrieve information about URL's again

[freddydk]

When you say that this worked with 24.5 - did you use another AAD App or did you not specify any scopes for New-BcAuthContext ? The default scope for New-BcAuthContext is api.businesscentral.dynamics.com/.default (which is for online) Since the audience validation fails - I assume your scope or AAD app is wrong?

[amea20]

Basically, nothing in the pipelines changed between Tuesday (current is 24.5) and Wednesday (current is 25.0), so it seems to be the BC25 has changed.

I am trying to help investigate where I can, and can see some differences between the Microsoft.Dynamics.Framework.UI.Client.dll files in C:\bcartifacts.cache\sandbox\25.0.23364.25044\platform\Test Assemblies and C:\bcartifacts.cache\sandbox\24.5.23489.25038\platform\Test Assemblies:

For instance, JsonHttpClient.cs in that dll file has a new AgentTaskIdentifierHeaderName header, not sure if it was intentional or for testing only.

Not sure if that is a factor at all, but there is definitely quite a few changes in that dll file, which is referenced in ClientContext.ps1 to help with authentication before running tests.

[freddydk]

I have a repro scenario here - will contact the team

[freddydk]

Apparently in BC25, there is a new setting in navsettings.json, which needs to be set to Valid Audience.

"AadValidAudience": "https://api.businesscentral.dynamics.com"

Will fix this in ContainerHelper.

[amea20]

Thanks for the quick response. I've added it as a workaround until the new version is released:

Invoke-ScriptInBcContainer -containerName bc -scriptblock {
    Set-NAVWebServerInstanceConfiguration -WebServerInstance BC -KeyName AadValidAudience -KeyValue 'https://api.businesscentral.dynamics.com'
}

[freddydk]

Fix is in BcContainerHelper preview as well.