Test Runner installed by default on 17.2.19367.20025-DK

[kaspermoerch]

Describe the issue Test Runner is installed by default when creating a new container using artifact version 17.2.19367.20025-DK. Pipelines now fail in DevOps because they try to install the Test Runner app. If the installation of the Test Runner app is removed from the pipeline then the compilation of the test app fails because the Test Runner app is not synchronized.

Scripts used to create container and cause the issue

$ArtifactUrl = Get-BCArtifactUrl -type Sandbox -country dk -version 17.2
$SecurePassword = ConvertTo-SecureString "P@ssW0rd" -AsPlainText -Force
$Credential = New-Object PSCredential ("admin", $SecurePassword)

$NewContainerArgs = @{ }
$NewContainerArgs += @{restart = "no" }
$NewContainerArgs += @{shortcuts = "None" }
$NewContainerArgs += @{Credential = $Credential }
$NewContainerArgs += @{isolation = "process" }
$NewContainerArgs += @{auth = "UserPassword" }
$NewContainerArgs += @{updateHosts = $true }
$NewContainerArgs += @{accept_outdated = $true }
$NewContainerArgs += @{containerName = "BC-17-CU2-DK" }
$NewContainerArgs += @{artifactUrl = $ArtifactUrl }
$NewContainerArgs += @{accept_eula = $true }
$NewContainerArgs += @{licenseFile = "***" }
$NewContainerArgs += @{useBestContainerOS = $true }

New-BcContainer @NewContainerArgs

Full output of scripts

BcContainerHelper is version 1.0.5
BcContainerHelper is not running as administrator
Host is Microsoft  - 1909
Docker Client Version is 20.10.0
Docker Server Version is 20.10.0
Fetching all docker images
Using image mcr.microsoft.com/dynamicsnav:10.0.18363.1198-generic
Creating Container BC-17-CU2-DK
Version: 17.2.19367.20025-DK
Style: sandbox
Multitenant: Yes
Platform: 17.0.19353.19941
Generic Tag: 0.1.0.24
Container OS Version: 10.0.18363.1198 (1909)
Host OS Version: 10.0.18363.1198 (1909)
Using process isolation
Using locale da-DK
Disabling the standard eventlog dump to container log every 2 seconds (use -dumpEventLog to enable)
Using license file ***
Files in C:\ProgramData\BcContainerHelper\Extensions\BC-17-CU2-DK\my:
- AdditionalOutput.ps1
- license.flf
- MainLoop.ps1
- SetupVariables.ps1
- updatehosts.ps1
Creating container BC-17-CU2-DK from image mcr.microsoft.com/dynamicsnav:10.0.18363.1198-generic
c2220389cd6f27c6a4a6d4a13232304cf1eea2747f1bef142167591949e466ba
Waiting for container BC-17-CU2-DK to be ready
Using artifactUrl https://bcartifacts.azureedge.net/sandbox/17.2.19367.20025/dk
Using installer from C:\Run\150-new
Installing Business Central
Installing from artifacts
Starting Local SQL Server
Starting Internet Information Server
Copying Service Tier Files
Copying PowerShell Scripts
Copying dependencies
Copying ReportBuilder
Importing PowerShell Modules
Determining Database Collation from c:\dl\sandbox\17.2.19367.20025\dk\BusinessCentral-DK.bak
Changing Database Server Collation to Danish_Greenlandic_100_CS_AS
Restoring CRONUS Demo Database
Exporting Application to CRONUS
Removing Application from tenant
Modifying Business Central Service Tier Config File for Docker
Creating Business Central Service Tier
Installing SIP crypto provider: 'C:\Windows\System32\NavSip.dll'
Copying Web Client Files
Copying Client Files
Copying ModernDev Files
Copying additional files
Copying ConfigurationPackages
Copying Test Assemblies
Copying Extensions
Copying Applications
Copying Applications.DK
Starting Business Central Service Tier
Importing license file
Copying Database on localhost\SQLEXPRESS from tenant to default
Taking database tenant offline
Copying database files
Attaching files as new Database default
Putting database tenant back online
Mounting tenant database
Mounting Database for default on server localhost\SQLEXPRESS with AllowAppDatabaseWrite = False
Sync'ing Tenant
Tenant is Operational
Stopping Business Central Service Tier
Installation took 278 seconds
Installation complete
Initializing...
Setting host.docker.internal to 192.168.86.52 in container hosts file (copy from host hosts file)
Setting gateway.docker.internal to 192.168.86.52 in container hosts file (copy from host hosts file)
Setting kubernetes.docker.internal to 127.0.0.1 in container hosts file (copy from host hosts file)
Setting host.containerhelper.internal to 172.30.224.1 in container hosts file
Starting Container
Hostname is BC-17-CU2-DK
PublicDnsName is BC-17-CU2-DK
Using NavUserPassword Authentication
Creating Self Signed Certificate
Self Signed Certificate Thumbprint AF99BB593BBDC7EEB8CBDE04D02B81A4700D93D0
Modifying Service Tier Config File with Instance Specific Settings
Starting Service Tier
Registering event sources
Creating DotNetCore Web Server Instance
Enabling Financials User Experience
Enabling rewrite rule: Don't rewrite system files
Enabling rewrite rule: Already have tenant specified
Enabling rewrite rule: Hostname (without port) to tenant
Using license file 'c:\run\my\license.flf'
Import License
Dismounting Tenant
Mounting Tenant
Mounting Database for default on server localhost\SQLEXPRESS with AllowAppDatabaseWrite = False
Sync'ing Tenant
Tenant is Operational
Creating http download site
Setting SA Password and enabling SA
Creating admin as SQL User and add to sysadmin
Creating SUPER user
Container IP Address: 172.30.238.69
Container Hostname  : BC-17-CU2-DK
Container Dns Name  : BC-17-CU2-DK
Web Client          : http://BC-17-CU2-DK/BC/?tenant=default
Dev. Server         : http://BC-17-CU2-DK
Dev. ServerInstance : BC
Dev. Server Tenant  : default
Setting BC-17-CU2-DK to 172.30.238.69 in host hosts file
Setting BC-17-CU2-DK-default to 172.30.238.69 in host hosts file

Files:
http://BC-17-CU2-DK:8080/ALLanguage.vsix

Container Total Physical Memory is 15.8Gb
Container Free Physical Memory is 3.7Gb

Initialization took 53 seconds
Ready for connections!
Reading CustomSettings.config from BC-17-CU2-DK
Container BC-17-CU2-DK successfully created

Use:
Get-BcContainerEventLog -containerName BC-17-CU2-DK to retrieve a snapshot of the event log from the container
Get-BcContainerDebugInfo -containerName BC-17-CU2-DK to get debug information about the container
Enter-BcContainer -containerName BC-17-CU2-DK to open a PowerShell prompt inside the container
Remove-BcContainer -containerName BC-17-CU2-DK to remove the container again
docker logs BC-17-CU2-DK to retrieve information about URL's again

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Running Get-NavAppInfo BC | Where-Object { $_.Name -match "Test Runner" } inside the container after creation renders the output:

Id: 23de40a6-dfe8-4f80-80db-d70f83ce8caf
Name: Test Runner
Version: 17.2.19367.20025
Publisher: Microsoft
ExtensionType: ModernDev
Scope: Global
Tenant:

The Test Runner app was not installed on versions lower than 17.2.

[waldo1001]

There also seems to be a difference between the OnPrem and Sandbox images .. still investigating though..

[kaspermoerch]

If I run Import-TestToolkitToBcContainer BC-17-CU2-DK -includeTestFrameworkOnly the output says:

Skipping app 'C:\Applications.DK\Microsoft_Test Runner_17.2.19367.20025.app' as it is already installed

Symbols are downloaded just fine, but when trying to publish my test app I get the error:

The request for path /BC/dev/apps?tenant=default&SchemaUpdateMode=recreate&DependencyPublishingOption=default failed with code UnprocessableEntity. Reason: Cannot synchronize the extension because no synchronized extension could be found to satisfy the dependency definition for Test Runner by Microsoft 17.0.0.0.

[freddydk]

I can acknowledge that the database in the artifacts indeed has the Test Runner and the Performance Toolkit installed. I will investigate why this is a problem - it should just skip the installation of that.

[freddydk]

I just tried to create a container and import test toolkit. It skips the Test Runner - but everything seems to work fine. @kaspermoerch you should make sure to update BcContainerHelper - I don't know if anything is different, but I do not see a problem with this at this time.

[freddydk]

Also - all my pipelines are not failing

[freddydk]

I think this is caused by the way ALOps installs the test apps. Have added a link to how I do it in containerhelper

[freddydk]

In 17.2, the performance toolkit and the test runner are pre-published in the database. BcContainerHelper will automatically check whether the test apps are installed before installing, but the generic image doesn't do this (since I thought I knew that apps couldn't be installed):-( if you add -myscripts @("https://raw.githubusercontent.com/microsoft/nav-docker/master/generic/Run/150-new/navinstall.ps1") to your new-bcimage call then it will grab the installer from the next generic version and use that.

It is not all though - it seems like the performance toolkit cannot be installed on Docker at all - there seem to be a check which disallows it to be installed on anything but onprem or online sandboxes - this check of course will need to change in the perf toolkit app - for now - avoid -includeperformancetoolkit on sandbox images.

[kaspermoerch]

Adding the installer to myscripts plus specifying -includeTestToolkit -includeTestFrameworkOnly when creating the container solves the problem.

[kaspermoerch]

In 17.2, the performance toolkit and the test runner are pre-published in the database. BcContainerHelper will automatically check whether the test apps are installed before installing, but the generic image doesn't do this (since I thought I knew that apps couldn't be installed):-( if you add -myscripts @("https://raw.githubusercontent.com/microsoft/nav-docker/master/generic/Run/150-new/navinstall.ps1") to your new-bcimage call then it will grab the installer from the next generic version and use that.

It is not all though - it seems like the performance toolkit cannot be installed on Docker at all - there seem to be a check which disallows it to be installed on anything but onprem or online sandboxes - this check of course will need to change in the perf toolkit app - for now - avoid -includeperformancetoolkit on sandbox images.

Will they be pre-published by default from now?

[freddydk]

Yes, I think we can safely assume so (at least that is what they told me)

[tfenster]

This is a quite ugly (breaking) change with a minor update. Can you share why this change happened? And do you see any chance to avoid that kind of break in the future?

[freddydk]

The reason I got was, that the app team wanted the performance toolkit to be preinstalled on sandbox tenants and since artifacts are built on the same database as we use for artifacts, then suddenly this happened.

It was a surprise for me as well, but I am not sure that we can avoid situations where the infrastructure team adds an app to the database, sorry.

[tfenster]

That part of the issue is not a problem, at least IMHO. We can just stop importing the testlibs. I was more worried by the fact that the performance toolkit can't be installed on an onprem container at all? Or does your comment say that this will be fixed?

[freddydk]

That will be fixed - that is just a bug. The code should check that the perf toolkit doesn't get installed on a production tenant. The test is wrong.

[tfenster]

ah perfect, thanks. Then it would be nice to know in advance instead of noticing through breaking pipelines, but not a real problem IMHO

[freddydk]

Test Runner is likely to be published in the database from 17.2 and forward. When using containerhelper, there was no problem installing the test toolkit with 17.2 even though Test Runner was there. When building an image (using generic image to install the test toolkit), it would fail due to the pre-existence of Test Runner.

The fix for this, all generic images:

SQL 2019: mcr.microsoft.com/businesscentral:{0}
SQL 2017: mcr.microsoft.com/businesscentral:{0}-sql2017

have been rebuilt with a fix to handle the existence of Test Runner in the database.

generic tag for SQL 2019 is 1.0.1.2 and for SQL 2017 is 0.1.0.26 (note the new naming of the sql2017 image)

Performance toolkit fix waits for a fix from the perf toolkit team in a future version of 17.2 likely. Note: perf toolkit will still works on onprem containers.

[tfenster]

@freddydk how do we get the fix? Just with the latest version of the artifacts?

[freddydk]

The perf toolkit will arrive with the artifacts when they have fixed this - until then - no perf toolkit on sandbox images. I assume that onprem works. The generic images have been updated and can handle the test runner.

[tfenster]

@freddydk do you know if the fix for the perf toolkit will appear with 17.3 sandbox images? We are still seeing it with generic 1.0.1.3 and 17.2 sandbox images

[freddydk]

Wasn't the issue with perf toolkit that you couldn't install it on onprem containers? I have not test 17.3 (was shipped today)

[tfenster]

@freddydk I assumed that it is the same issue, but I don't have the old logs anymore, so I can't say for sure. I have created #1617