Run-TestsInBCContainer results in 'ClientSession State is InError'

[favmac]

Describe the issue When I execute the Run-TestInBCContainer command in a new blank BC16-Container, there are two test codeunits (138696 and 138697) which make the script result in an error. Test execution of the same codeunits in the TestTool of the WebClient is successful.

Script used to create container

# Runtime
$ContainerName = 'BC16Test'

# Artifact Information
$ArtifactCountry = "w1"
$ArtifactVersion = "16.5"
$ArtifactSelect = "Latest"
$ArtifactType = "onprem"
$ArtifactURL = Get-BCArtifactUrl -type $ArtifactType -select $ArtifactSelect -country $ArtifactCountry -version $ArtifactVersion

# License
$LicensePathFileName = "C:\License\License.flf"

# User
$Username = 'test'
$Password = 'test'
$SecurePassword = ConvertTo-SecureString -String $Password -AsPlainText -Force
$Credentials = new-object -typename System.Management.Automation.PSCredential -argumentlist $Username, $SecurePassword

$myscripts = @(@{'MainLoop.ps1' = 'while ($true) { start-sleep -seconds 10 }' })

New-BCContainer @parameters `
    -accept_eula:$true `
    -containerName $ContainerName `
    -artifactUrl $ArtifactURL `
    -auth NAVUserPassword `
    -Credential $Credentials `
    -updateHosts `
    -doNotExportObjectsToText `
    -MemoryLimit $memoryLimit `
    -shortcuts DesktopFolder `
    -includeTestToolkit $true `
    -enableTaskScheduler:$false `
    -accept_outdated:$true `
    -useBestContainerOS:$true `
    -licenseFile $LicensePathFileName `
    -imageName fav `
    -multitenant:$false `
    -myScripts $myscripts `
    -locale $Locale

Full output of script

BcContainerHelper version 1.0.16
BcContainerHelper is version 1.0.16
BcContainerHelper is running as administrator
Host is Microsoft  - 1909
Docker Client Version is 19.03.12
Docker Server Version is 19.03.12
Removing container BC16Test
Removing BC16Test from host hosts file
Removing BC16Test-* from host hosts file
Removing C:\ProgramData\BcContainerHelper\Extensions\BC16Test
Fetching all docker images
ArtifactUrl and ImageName specified
Image fav:onprem-16.5.15897.15953-w1 already exists
Using image fav:onprem-16.5.15897.15953-w1
Creating Container BC16Test
Version: 16.5.15897.15953-w1
Style: onprem
Multitenant: No
Platform: 16.0.15884.15941
Generic Tag: 1.0.1.2
Container OS Version: 10.0.18363.1256 (1909)
Host OS Version: 10.0.18363.1256 (1909)
Using process isolation
Using locale 
Disabling the standard eventlog dump to container log every 2 seconds (use -dumpEventLog to enable)
Using license file C:\License\License.flf
Additional Parameters:
--env customNavSettings=EnableTaskScheduler=False
Files in C:\ProgramData\BcContainerHelper\Extensions\BC16Test\my:
- AdditionalOutput.ps1
- license.flf
- MainLoop.ps1
- SetupVariables.ps1
- updatehosts.ps1
Creating container BC16Test from image fav:onprem-16.5.15897.15953-w1
a794193a930d8c67641779106df9462ae158d4bff46225797f14e04ef6796f7c
Waiting for container BC16Test to be ready
Initializing...
Setting host.docker.internal to 192.168.178.22 in container hosts file (copy from host hosts file)
Setting gateway.docker.internal to 192.168.178.22 in container hosts file (copy from host hosts file)
Setting kubernetes.docker.internal to 127.0.0.1 in container hosts file (copy from host hosts file)
Setting host.containerhelper.internal to 172.17.64.1 in container hosts file
Starting Container
Hostname is BC16Test
PublicDnsName is BC16Test
Using NavUserPassword Authentication
Starting Local SQL Server
Starting Internet Information Server
Creating Self Signed Certificate
Self Signed Certificate Thumbprint C4C514E672D1C1C266759D6988A7730AA0CBB57E
Modifying Service Tier Config File with Instance Specific Settings
Modifying Service Tier Config File with settings from environment variable
Setting EnableTaskScheduler to False
Starting Service Tier
Registering event sources
Creating DotNetCore Web Server Instance
Using license file 'c:\run\my\license.flf'
Import License
Creating http download site
Setting SA Password and enabling SA
Creating test as SQL User and add to sysadmin
Creating SUPER user
WARNING: The password that you entered does not meet the minimum requirements. 
It should be at least 8 characters long and contain at least one uppercase 
letter, one lowercase letter, and one number.
Container IP Address: 172.17.65.208
Container Hostname  : BC16Test
Container Dns Name  : BC16Test
Web Client          : http://BC16Test/BC/
Dev. Server         : http://BC16Test
Dev. ServerInstance : BC
Setting BC16Test to 172.17.65.208 in host hosts file

Files:
http://BC16Test:8080/ALLanguage.vsix

Container Total Physical Memory is 15.4Gb
Container Free Physical Memory is 4.4Gb

Initialization took 122 seconds
Ready for connections!
Reading CustomSettings.config from BC16Test
Creating Desktop Shortcuts for BC16Test
Copy from container BC16Test (C:\Applications\TestFramework\TestLibraries\Any\Microsoft_Any.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Micro
soft_Any.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Any.app
Synchronizing Any on tenant default
Installing Any on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\TestFramework\TestLibraries\Assert\Microsoft_Library Assert.app) to C:\ProgramData\BcContainerHelper\Extensions\B
C16Test\_Microsoft_Library Assert.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Library Assert.app
Synchronizing Library Assert on tenant default
Installing Library Assert on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\TestFramework\TestRunner\Microsoft_Test Runner.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Micr
osoft_Test Runner.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Test Runner.app
Synchronizing Test Runner on tenant default
Installing Test Runner on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\system application\test\Microsoft_System Application Test Library.app) to C:\ProgramData\BcContainerHelper\Extens
ions\BC16Test\_Microsoft_System Application Test Library.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_System Application Test Library.app
Synchronizing System Application Test Library on tenant default
Installing System Application Test Library on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-TestLibraries.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsof
t_Tests-TestLibraries.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-TestLibraries.app
Synchronizing Tests-TestLibraries on tenant default
Installing Tests-TestLibraries on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Bank.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-B
ank.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Bank.app
Synchronizing Tests-Bank on tenant default
Installing Tests-Bank on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Cash Flow.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Te
sts-Cash Flow.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Cash Flow.app
Synchronizing Tests-Cash Flow on tenant default
Installing Tests-Cash Flow on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Cost Accounting.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Micros
oft_Tests-Cost Accounting.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Cost Accounting.app
Synchronizing Tests-Cost Accounting on tenant default
Installing Tests-Cost Accounting on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-CRM integration.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Micros
oft_Tests-CRM integration.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-CRM integration.app
Synchronizing Tests-CRM integration on tenant default
Installing Tests-CRM integration on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Data Exchange.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsof
t_Tests-Data Exchange.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Data Exchange.app
Synchronizing Tests-Data Exchange on tenant default
Installing Tests-Data Exchange on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Dimension.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Te
sts-Dimension.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Dimension.app
Synchronizing Tests-Dimension on tenant default
Installing Tests-Dimension on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-ERM.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-ER
M.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-ERM.app
Synchronizing Tests-ERM on tenant default
Installing Tests-ERM on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Fixed Asset.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_
Tests-Fixed Asset.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Fixed Asset.app
Synchronizing Tests-Fixed Asset on tenant default
Installing Tests-Fixed Asset on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-General Journal.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Micros
oft_Tests-General Journal.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-General Journal.app
Synchronizing Tests-General Journal on tenant default
Installing Tests-General Journal on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Graph.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-
Graph.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Graph.app
Synchronizing Tests-Graph on tenant default
Installing Tests-Graph on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Integration.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_
Tests-Integration.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Integration.app
Synchronizing Tests-Integration on tenant default
Installing Tests-Integration on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Invoicing.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Te
sts-Invoicing.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Invoicing.app
Synchronizing Tests-Invoicing on tenant default
Installing Tests-Invoicing on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Job.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Jo
b.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Job.app
Synchronizing Tests-Job on tenant default
Installing Tests-Job on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Local.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-
Local.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Local.app
Synchronizing Tests-Local on tenant default
Installing Tests-Local on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Misc.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-M
isc.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Misc.app
Synchronizing Tests-Misc on tenant default
Installing Tests-Misc on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Permissions.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_
Tests-Permissions.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Permissions.app
Synchronizing Tests-Permissions on tenant default
Installing Tests-Permissions on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Physical Inventory.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Mic
rosoft_Tests-Physical Inventory.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Physical Inventory.app
Synchronizing Tests-Physical Inventory on tenant default
Installing Tests-Physical Inventory on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Prepayment.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_T
ests-Prepayment.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Prepayment.app
Synchronizing Tests-Prepayment on tenant default
Installing Tests-Prepayment on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Rapid Start.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_
Tests-Rapid Start.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Rapid Start.app
Synchronizing Tests-Rapid Start on tenant default
Installing Tests-Rapid Start on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Report.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests
-Report.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Report.app
Synchronizing Tests-Report on tenant default
Installing Tests-Report on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Resource.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tes
ts-Resource.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Resource.app
Synchronizing Tests-Resource on tenant default
Installing Tests-Resource on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Reverse.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Test
s-Reverse.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Reverse.app
Synchronizing Tests-Reverse on tenant default
Installing Tests-Reverse on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-SCM.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-SC
M.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-SCM.app
Synchronizing Tests-SCM on tenant default
Installing Tests-SCM on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-SMB.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-SM
B.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-SMB.app
Synchronizing Tests-SMB on tenant default
Installing Tests-SMB on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-SMTP.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-S
MTP.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-SMTP.app
Synchronizing Tests-SMTP on tenant default
Installing Tests-SMTP on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Upgrade.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Test
s-Upgrade.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Upgrade.app
Synchronizing Tests-Upgrade on tenant default
Installing Tests-Upgrade on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-User.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-U
ser.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-User.app
Synchronizing Tests-User on tenant default
Installing Tests-User on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-VAT.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-VA
T.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-VAT.app
Synchronizing Tests-VAT on tenant default
Installing Tests-VAT on tenant default
App successfully published
Copy from container BC16Test (C:\Applications\BaseApp\Test\Microsoft_Tests-Workflow.app) to C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tes
ts-Workflow.app
Publishing C:\ProgramData\BcContainerHelper\Extensions\BC16Test\_Microsoft_Tests-Workflow.app
Synchronizing Tests-Workflow on tenant default
Installing Tests-Workflow on tenant default
App successfully published
TestToolkit successfully imported
Container BC16Test successfully created

...

Script that causes the issue

$ContainerName = 'BC16Test'

$Username = 'test'
$Password = 'test'
$SecurePassword = ConvertTo-SecureString -AsPlainText $Password -Force
$Credentials = new-object -typename System.Management.Automation.PSCredential -argumentlist $Username, $SecurePassword
[timespan] $interactionTimeout = [timespan]::FromHours(24)

#Import Tests
$apps = Get-NavContainerAppInfo -containerName $ContainerName -tenant "default" -tenantSpecificProperties | where {$_.Name -match "Tests-*"}

foreach ($app in $apps) {
    $TempTestResultFile = "C:\ProgramData\BcContainerHelper\Extensions\$ContainerName\my\TEST-$($App.Name).xml"
    $tests = Get-TestsFromBCContainer `
        -containerName $containerName `
        -credential $Credentials `
        -extensionId $app.AppId `
        -ignoreGroups
    $first = $true
    $tests | ? {$_.Id -in '138696','138697'} | ForEach-Object {
        Run-TestsInBCContainer `
            -containerName $containerName `
            -credential $Credentials `
            -detailed `
            -XUnitResultFileName $TempTestResultFile `
            -AppendToXUnitResultFile:(!$first) `
            -testCodeunit $_.Id `
            -interactionTimeout $interactionTimeout
        $first = $false
    }
}

Full output of script

  Codeunit 138696 Page Troubleshooting Test UnhandledException : Object reference not set to an instance of an object.
at <ScriptBlock>, C:\ProgramData\BcContainerHelper\Extensions\BC16Test\PsTestTool\ClientContext.ps1: line 117
at <ScriptBlock>, <No file>: line 76
Current Interaction: Microsoft.Dynamics.Framework.UI.Client.Interactions.InvokeActionInteraction
Time spend: 3 seconds
at AwaitState, C:\ProgramData\BcContainerHelper\Extensions\BC16Test\PsTestTool\ClientContext.ps1: line 217
at InvokeInteraction, C:\ProgramData\BcContainerHelper\Extensions\BC16Test\PsTestTool\ClientContext.ps1: line 236
at InvokeAction, C:\ProgramData\BcContainerHelper\Extensions\BC16Test\PsTestTool\ClientContext.ps1: line 453
at Run-Tests, C:\ProgramData\BcContainerHelper\Extensions\BC16Test\PsTestTool\PsTestFunctions.ps1: line 805
at <ScriptBlock>, <No file>: line 57
ClientSession State is InError (Wait time 3 seconds)
at AwaitState, C:\ProgramData\BcContainerHelper\Extensions\BC16Test\PsTestTool\ClientContext.ps1: line 217
at InvokeInteraction, C:\ProgramData\BcContainerHelper\Extensions\BC16Test\PsTestTool\ClientContext.ps1: line 236
at InvokeAction, C:\ProgramData\BcContainerHelper\Extensions\BC16Test\PsTestTool\ClientContext.ps1: line 453
at Run-Tests, C:\ProgramData\BcContainerHelper\Extensions\BC16Test\PsTestTool\PsTestFunctions.ps1: line 805
at <ScriptBlock>, <No file>: line 57
ClientSession State is InError (Wait time 3 seconds)
In C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\1.0.16\AppHandling\Run-TestsInNavContainer.ps1:443 Zeichen:17
+                 throw $_.Exception.Message
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (ClientSession S...time 3 seconds):String) [], RuntimeException
    + FullyQualifiedErrorId : ClientSession State is InError (Wait time 3 seconds)

Screenshots

Additional context

• the script works for all other standard test codeunits excluding 138696 and 138697
• Error appears only when using Run-TestsInBCContainer (not in the WebClient TestTool)

[freddydk]

Did you try to use the extension id on run-tests instead of getting the test codeunits and running these?

[favmac]

If you mean this:

the result is the same

[freddydk]

remove the -testcodeunit and the loop on tests - one call to run-tests per app

[favmac]

Is there another way to restrict the codeunits for reducing the runtime than the testcodeunit parameter?

[freddydk]

use disabledtests.json

[favmac]

OK. That's it. With a big DisabledTests.json including all Test Codeunits except the problematic it was successful.

Thank you!