Performance degree between BC23 and BC24

[RonKoppelaar]

I noticed a performance degradation in the build pipeline moving to BC24. If I look at the compile and publish steps these are the time used in minutes and seconds.

BC24 - https://bcartifacts.azureedge.net/sandbox/24.0.16410.18040/nl Compile: 12:13 (mm:ss) Publish: 8:34 (mm:ss)

BC23.5 - https://bcartifacts.azureedge.net/sandbox/23.5.16502.16887/nl Compile 8:18 (mm:ss) Publish: 3:58 (mm:ss)

Both pipelines compile and publish the same amount of apps using same build scripts

Buildservers are running with W2019 and uses Process Isolation. Do you know there are known issues? According to things I read new platform is based on .Net8 which should actually be faster then .Net6

For Compile and publish I uses the std. cmdlets from BCContainer: Compile-AppInNavContainer Publish-NavContainerApp

BcContainerHelper is version 6.0.15

[freddydk]

Have you set usePwshForBC24 to true or false in settings? (or are you using the default value for this)

[RonKoppelaar]

No not in those scripts.. Later in the pipeline I do. Basically using the default

[freddydk]

Do you have detailed logs of before and after with timestamps?

[RonKoppelaar]

Please find attached logfiles LogFiles.zip

It contains the raw logfile output from Azure devops for prepare and compile step. There is a slide difference in prepare step between BC23 and BC24 For BC23

• Start with empty DB and publish the MS apps we need

For BC23

• Start with Cronus container
• Remove apps we dont need

Beside this all should be the same.

[RonKoppelaar]

I'll also add the comparison for PostBuild. Also there huge difference, But I expect I used UsePwsBC24 there.

PostBuildLogs.zip

PostBuild is for creating bacpacs PostbuildOnprem recreates a container by publishing the compiled apps and creating runtime packages.

[MattTraxinger]

Just confirming that I have also noticed this over the past two weeks. Our pipelines that used compiler folders were clocking in around 2 minutes, now they are closer to 4 and 5. Full container builds are seeing roughly the same increase is time. It's not just you, but I don't know if it was a BCContainerHelper change or a BC change.

[marknitek]

The whole -usePwsh thing new to me, so maybe this is unrelated but when running simple commands with invoke-scriptinbccontainer it will generally take 2x as long with PS7 compared to using PS5 (-usePwsh:$false)

Measure-Command { Invoke-ScriptInBcContainer $containerName -scriptblock { $config = Get-Navserverconfiguration $ServerInstance} -usePwsh:$false }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 833
Ticks             : 8335212
TotalDays         : 9.64723611111111E-06
TotalHours        : 0.000231533666666667
TotalMinutes      : 0.01389202
TotalSeconds      : 0.8335212
TotalMilliseconds : 833.5212

Measure-Command { Invoke-ScriptInBcContainer $containerName -scriptblock { $config = Get-Navserverconfiguration $ServerInstance} -usePwsh:$true } 

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 1
Milliseconds      : 639
Ticks             : 16392312
TotalDays         : 1.89725833333333E-05
TotalHours        : 0.000455342
TotalMinutes      : 0.02732052
TotalSeconds      : 1.6392312
TotalMilliseconds : 1639.2312

These were measurements on a BC23 container.

In BC24 i'am unable to get these results (800ms) :

Measure-Command { Invoke-ScriptInBcContainer test24 -scriptblock { $config = Get-Navserverconfiguration $ServerInstance} -usePwsh:$false }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 1
Milliseconds      : 361
Ticks             : 13618158
TotalDays         : 1.57617569444444E-05
TotalHours        : 0.000378282166666667
TotalMinutes      : 0.02269693
TotalSeconds      : 1.3618158
TotalMilliseconds : 1361.8158 

[marknitek]

This is true for any container by the way because of the version check which adds the 800ms to any command executed by invoke-scriptinbccontainer:

https://github.com/microsoft/navcontainerhelper/blob/f157a40f342e08a9e8b3534de933b84a093ae83b/ContainerHandling/Invoke-ScriptInNavContainer.ps1#L37

[freddydk]

and if you set $bcContainerHelperConfig.usePwshForBC24 = $false - then everything is back to normal? The reason for this was that all PowerShell cmdlets in BC24 are PowerShell 7 and the bridge also added quite a big overhead. Therefore I created this and defaulted that to true. Let me hear what the results are for the above and then figure out what the best approach is

[RonKoppelaar]

Freddy according to your question, is it related to my Initial problem? The build uses the default cmdlet to create the container,, without setting the $bcContainerHelperConfig.usePwshForBC24. Next it will compile and publish using the bccontainerhelper cmdlets. Basicly both steps are significant slower. But mainly the publish step.

[marknitek]

and if you set $bcContainerHelperConfig.usePwshForBC24 = $false - then everything is back to normal? The reason for this was that all PowerShell cmdlets in BC24 are PowerShell 7 and the bridge also added quite a big overhead. Therefore I created this and defaulted that to true. Let me hear what the results are for the above and then figure out what the best approach is

If that was question was related to my comments: setting it to $false does the same effect as using -usepwsh:$false (as you would expect) and lowers execution times for Versions. But BC24 is still slower then BC23 (~800ms compared to ~1400ms). I have no idea where these differences come from...

@RonKoppelaar interesting findings, i checked by entering two containers (BC23/BC24) and running a simple Get-NAVServerInstance:

BC23

 Measure-Command { Get-NAVServerConfiguration $ServerInstance }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 72
Ticks             : 720100
TotalDays         : 8.33449074074074E-07
TotalHours        : 2.00027777777778E-05
TotalMinutes      : 0.00120016666666667
TotalSeconds      : 0.07201
TotalMilliseconds : 72.01

BC24

 Measure-Command { Get-NAVServerConfiguration $ServerInstance }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 666
Ticks             : 6660190
TotalDays         : 7.70855324074074E-06
TotalHours        : 0.000185005277777778
TotalMinutes      : 0.0111003166666667
TotalSeconds      : 0.666019
TotalMilliseconds : 666.019

Huge difference! is this comming from the ps5 -> ps7 wrapper?

[freddydk]

@marknitek Yes, this is the reason why I default containers to run PowerShell 7 by default. But that comes with another cost - PowerShell 7 cannot (as far as I know) do PowerShell remoting - I cannot create a session - I need to use docker exec, which then has a similar cost - but maybe we can do something about that... Will have to investigate that.

[marknitek]

@freddydk i tested with both now:

$bcContainerHelperConfig.usePwshForBC24 = $true
Write-Host "Total Execution: "(Measure-Command { 
    Invoke-ScriptInBcContainer test24 -scriptblock { 
        Write-Host ($PSVersionTable | Out-String);
        Write-Host "Get-NavServerConfiguration: " (Measure-Command { Get-Navserverconfiguration $ServerInstance}).TotalSeconds "seconds"
    } 
}).TotalSeconds "seconds"

Name                           Value
----                           -----
PSVersion                      7.4.1
PSEdition                      Core
GitCommitId                    7.4.1
OS                             Microsoft Windows 10.0.22000
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Get-NavServerConfiguration:  0.1663978 seconds
Total Execution:  2.6827118 seconds

$bcContainerHelperConfig.usePwshForBC24 = $false
Write-Host "Total Execution: "(Measure-Command { 
    Invoke-ScriptInBcContainer test24 -scriptblock { 
        Write-Host ($PSVersionTable | Out-String);
        Write-Host "Get-NavServerConfiguration: " (Measure-Command { Get-Navserverconfiguration $ServerInstance}).TotalSeconds "seconds"
    } 
}).TotalSeconds "seconds"

Name                           Value
----                           -----
PSVersion                      5.1.20348.2227
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.20348.2227
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Get-NavServerConfiguration:  0.7761208 seconds
Total Execution:  1.5641623 seconds

[marknitek]

@freddydk regarding psremoting in ps7. I tinkered around a bit and i think it works quite well but must be enabled with Enable-PSRemoting in the pwsh first.

I had a look at Get-NavContainerSession and used these commands to get a session but for Powershell7:

$session = New-PSSession -Credential $credential -ComputerName $containerName -Authentication Basic -useSSL -SessionOption (New-PSSessionOption -SkipCACheck -SkipCNCheck) -ConfigurationName "Powershell.7" 

But that failed at first because there was no configuraiton for "Powershell.7". Then i enabled PSRemoting with:

Invoke-ScriptInBcContainer $containerName -scriptblock { Enable-PSRemoting;Get-PSSessionConfiguration }

Name          : PowerShell.7
PSVersion     : 7.4
StartupScript :
RunAsUser     :
Permission    : NT AUTHORITY\INTERACTIVE AccessAllowed, BUILTIN\Administrators AccessAllowed, BUILTIN\Remote
                Management Users AccessAllowed

Name          : PowerShell.7.4.1
PSVersion     : 7.4
StartupScript :
RunAsUser     :
Permission    : NT AUTHORITY\INTERACTIVE AccessAllowed, BUILTIN\Administrators AccessAllowed, BUILTIN\Remote
                Management Users AccessAllowed

Then i was able to use the session:

$session = New-PSSession -Credential $credential -ComputerName $containerName -Authentication Basic -useSSL -SessionOption (New-PSSessionOption -SkipCACheck -SkipCNCheck) -ConfigurationName "Powershell.7"
Invoke-Command -Session $session -ScriptBlock { $PSVersionTable | Out-String }

Name                           Value
----                           -----
PSVersion                      7.4.1
PSEdition                      Core
GitCommitId                    7.4.1
OS                             Microsoft Windows 10.0.22000
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion
SerializationVersion           1.1.0.1
WSManStackVersion              3.0 

[freddydk]

That is great news, thanks for investigating. I will add this to containerhelper and assume that we will see performance increases after this

[freddydk]

@marknitek what did you use as credential? Did you create a windows user inside the container?

[freddydk]

It looks like you are using Windows Authentication, right?

[marknitek]

@freddydk i did not create anything, i just used the code that was present in Get-NavContainerSession to get the credentials which previously created all this when using remoting with regular powershell. Imo it was just a matter of Enabling remoting in the pwsh instance.

But the code you have in place for generating all this still remains valid and works across powershell and pwsh:

$UUID = (Get-CimInstance win32_ComputerSystemProduct).UUID
                $credential = New-Object PSCredential -ArgumentList 'winrm', (ConvertTo-SecureString -string $UUID -AsPlainText -force)
                Invoke-ScriptInBcContainer -containerName $containerName -useSession:$false -scriptblock { Param([PSCredential] $credential)
                    $winrmuser = get-localuser -name $credential.UserName -ErrorAction SilentlyContinue
                    if (!$winrmuser) {
                        $cert = New-SelfSignedCertificate -DnsName "dontcare" -CertStoreLocation Cert:\LocalMachine\My
                        winrm create winrm/config/Listener?Address=*+Transport=HTTPS ('@{Hostname="dontcare"; CertificateThumbprint="' + $cert.Thumbprint + '"}') | Out-Null
                        winrm set winrm/config/service/Auth '@{Basic="true"}' | Out-Null
                        Write-Host "`nCreating Container user $($credential.UserName)"
                        New-LocalUser -AccountNeverExpires -PasswordNeverExpires -FullName $credential.UserName -Name $credential.UserName -Password $credential.Password | Out-Null
                        Add-LocalGroupMember -Group administrators -Member $credential.UserName | Out-Null
                    }
                } -argumentList $credential
                $session = New-PSSession -Credential $credential -ComputerName $containerName -Authentication Basic -useSSL -SessionOption (New-PSSessionOption -SkipCACheck -SkipCNCheck)

[freddydk]

Got it - that was what I was after - it creates a local administrator in the container which it can use for this. I thought you found a different way:-)

[RonKoppelaar]

@freddydk is there anything I can do to further test/validate performance issues which could help for a resolution?

[freddydk]

I will modify the Invoke-ScriptInBcContainer to use sessions in all combinations (ps5 -> ps5, ps5 -> ps7, ps7 -> ps5 and ps7 -> ps7 - when that is done - I would like some tests and specific issues on which things might still be slow.

[freddydk]

@marknitek what OS are you running? I am having issues with this on my Windows 11 machine - the winrm stuff doesn't work at all...:-( WIll need to check some more

[freddydk]

Please try the ContainerHelper changes from this PR: https://github.com/microsoft/navcontainerhelper/pull/3496

and give me some feedback on which changes you see. With this, I can create PS5 and PS7 sessions in the container from both PS5 and PS7 (admin or non-admin) on the host.

If a Session is created from PS5 in admin mode, the New-PSSession with the ContainerId parameter is used - no credentials are needed. If a session is created from PS7 or PS5 in non-admin mode, then winrm is used. For winrm, BcContainerHelper automatically creates a user inside the container with username = winrm and the password is the UUID of the host computer. If the config setting useSslForWinRmSession is true (default), it uses HTTPS for the container connection (port 5986). If the config setting useSslForWinRmSession is false, it uses HTTP for the container connection (port 5985).

Invoke-scriptInBcContainer -containerName $containerName -scriptblock { $PSVersionTable | out-host }

should show the PS version used in the container. It takes a little time to create the session - but subsequent calls should be fast as the session is cached.

On my machine, the first invoke to create the winrm session takes 1-2 seconds (unless I am running PS5 as admin) and after that - every call takes ~300ms

[freddydk]

Here are some examples of running the new session mechanisms in the different modes, which differ. The script I am running is basically showing the PowerShell version inside the container, the username inside the container and the number of milliseconds it took to run the script

[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc23 -useSession $true -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds
[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc23 -useSession $true -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds
[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc23 -useSession $true -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds
[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc24 -useSession $true -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds
[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc24 -useSession $true -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds
[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc24 -useSession $true -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds

Running in PS5 in admin mode

PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
2193
PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
5456
PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
796
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
4906
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
809
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
794

Running in PS5 in non-admin mode

PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
1383
PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
5404
PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
760
PowerShell Version: 7.4.1
Username: bc24\winrm
2965
PowerShell Version: 7.4.1
Username: bc24\winrm
513
PowerShell Version: 7.4.1
Username: bc24\winrm
500

Running in PS7 in admin mode

PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
1433
PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
5420
PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
750
PowerShell Version: 7.4.1
Username: bc24\winrm
4277
PowerShell Version: 7.4.1
Username: bc24\winrm
699
PowerShell Version: 7.4.1
Username: bc24\winrm
659

Running in PS7 in non-admin mode

PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
1595
PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
5836
PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
707
PowerShell Version: 7.4.1
Username: bc24\winrm
4093
PowerShell Version: 7.4.1
Username: bc24\winrm
772
PowerShell Version: 7.4.1
Username: bc24\winrm
777

In interesting observation is, that the second time we run a PS5 session inside the container (where the session is cached), it for some reasons takes much longer than the first time - I have no idea why this happens. Subsequent invokes are all faster - around 800ms.

Note that when running any combination beside PS5 in admin mode, the user is a newly created local admin inside the container called winrm with a hardcoded password set to the UUID of the host computer.

On my machine - SSL connection to winrm doesn't work at all - I have to set:

$bcContainerHelperConfig.useSslForWinRmSession = $false

If you want to always use WinRm (also on PS5 in admin mode) you can set:

$bcContainerHelperConfig.alwaysUseWinRmSession = $true

This isn't set by default as this would force everybody to always use new behavior - and with the number of people using BcContainerHelper, I know that any change causes disruption.

Setting useSession to false will cause Invoke-ScriptInBcContainer to use docker exec when running scripts inside the container. The script looks then like this:

[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc23 -useSession $false -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds
[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc23 -useSession $false -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds
[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc23 -useSession $false -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds
[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc24 -useSession $false -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds
[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc24 -useSession $false -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds
[int](Measure-Command {Invoke-ScriptInBcContainer -containerName bc24 -useSession $false -scriptblock { Write-Host "PowerShell Version: $($PSVersionTable.PSVersion)"; Write-Host "Username: $(whoami)" }}).TotalMilliseconds

Running in PS5 in admin mode

PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1986
PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1455
PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1451
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1503
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1505
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1505

Running in PS5 in non-admin mode

PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1938
PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1342
PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1351
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1403
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1402
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1430

Running in PS7 in admin mode

PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1621
PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1083
PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1092
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1156
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1149
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1161

Running in PS7 in non-admin mode

PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1076
PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1055
PowerShell Version: 5.1.20348.2400
Username: user manager\containeradministrator
1073
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1141
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1110
PowerShell Version: 7.4.1
Username: user manager\containeradministrator
1135

Using docker exec for script invoke inside the container are (on my machine) fairly consistent and the first call is faster then creating a winrm session.

Things to investigate:

1. Why is the second script invoke into a PS5 session inside the container waaaaay slower than the first and the third
2. Are there any things that the winrm user cannot do inside the container (or stuff that works differently)
3. Why can my machine not run winrm over SSL using a self-signed certificate inside the container (it works in GitHub hosted runners just fine...). Please let me know if other people are having this issue as well.

[freddydk]

Wanting to investigate no. 1 from above, and the behavior disappeared without any changes... Now the time is fairly consistent like this:

PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
1151
PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
412
PowerShell Version: 5.1.20348.2400
Username: bc23\winrm
412
PowerShell Version: 7.4.1
Username: bc24\winrm
1267
PowerShell Version: 7.4.1
Username: bc24\winrm
401
PowerShell Version: 7.4.1
Username: bc24\winrm
393

Will not investigate further unless other people are reporting stranger things...

[freddydk]

Found out that the whoami for some reason some times takes a lot of time (5 seconds) - replaced this with $env:username

[freddydk]

Created new containers and running without using whoami

Running in PS5 in admin mode (with alwaysUseWinRmSession = false)

PowerShell Version: 5.1.20348.2400
Username: ContainerAdministrator
2654
PowerShell Version: 5.1.20348.2400
Username: ContainerAdministrator
524
PowerShell Version: 5.1.20348.2400
Username: ContainerAdministrator
522
PowerShell Version: 7.4.1
Username: ContainerAdministrator
3523
PowerShell Version: 7.4.1
Username: ContainerAdministrator
530
PowerShell Version: 7.4.1
Username: ContainerAdministrator
505

Running in PS5 in admin mode (with alwaysUseWinRmSession = true)

PowerShell Version: 5.1.20348.2400
Username: winrm
1980
PowerShell Version: 5.1.20348.2400
Username: winrm
535
PowerShell Version: 5.1.20348.2400
Username: winrm
529
PowerShell Version: 7.4.1
Username: winrm
2135
PowerShell Version: 7.4.1
Username: winrm
538
PowerShell Version: 7.4.1
Username: winrm
534

Running in PS5 in non-admin mode

PowerShell Version: 5.1.20348.2400
Username: winrm
1331
PowerShell Version: 5.1.20348.2400
Username: winrm
486
PowerShell Version: 5.1.20348.2400
Username: winrm
469
PowerShell Version: 7.4.1
Username: winrm
2041
PowerShell Version: 7.4.1
Username: winrm
489
PowerShell Version: 7.4.1
Username: winrm
476

Running in PS7 in admin mode

PowerShell Version: 5.1.20348.2400
Username: winrm
1095
PowerShell Version: 5.1.20348.2400
Username: winrm
362
PowerShell Version: 5.1.20348.2400
Username: winrm
367
PowerShell Version: 7.4.1
Username: winrm
1967
PowerShell Version: 7.4.1
Username: winrm
376
PowerShell Version: 7.4.1
Username: winrm
356

Running in PS7 in non-admin mode

PowerShell Version: 5.1.20348.2400
Username: winrm
1057
PowerShell Version: 5.1.20348.2400
Username: winrm
365
PowerShell Version: 5.1.20348.2400
Username: winrm
351
PowerShell Version: 7.4.1
Username: winrm
1748
PowerShell Version: 7.4.1
Username: winrm
369
PowerShell Version: 7.4.1
Username: winrm
360

Things still to investigate:

1. Why is the second script invoke into a PS5 session inside the container waaaaay slower than the first and the third Are there any things that the winrm user cannot do inside the container (or stuff that works differently)
2. Why can my machine not run winrm over SSL using a self-signed certificate inside the container (it works in GitHub hosted runners just fine...). Please let me know if other people are having this issue as well.

[RonKoppelaar]

@freddydk lot of investigations regarding session management. Is there anything I can already test in my build containers regarding the compile and publish step to see any improvements?

[freddydk]

Are you able to test containerhelper from a branch? Then you can try the branch above - this should cause all switching between ps5 and ps7 to be gone

[marknitek]

@freddydk looks good to me. I can confirm that i now get consistent low execution times (on subsequent runs) for both containers (23 and 24 - same that i used for the above tests). PS7 is used for the 24 container and PS5 for the 23 container as expected.

I did not change anything in the config to achieve that. Execution time is about 700ms. I will have to do some more tests to say anything about reliability of the winrm session. But i guess it should work fine as long as the container user is a local admin? I can not think of anything that would be bound to the container user context

[freddydk]

@MattTraxinger you mentioned that CompilerFolder compilations also had degrading performance - are you using PS5 or PS7 and does it make a difference if you change?

[freddydk]

@MattTraxinger just tried one of my repositories with 5 projects using CompilerFolder functionality in AL-Go for GitHub. The Build step has the following timings:

Windows/Pwsh (before 24.0)
1:39 | 1:21
1:31 | 1:30
1:32 | 1:35
1:48 | 1:53
1:22 | 1:29
-----------
1:34 | 1:34

Windows/PowerShell (24.0)
1:26 | 1:53
1:24 | 1:31
1:25 | 1:43
1:35 | 1:38
1:38 | 1:38
-----------
1:30 | 1:41

Windows/Pwsh (24.0)
1:35 | 1:27
1:28 | 1:10
1:15 | 1:37
1:28 | 1:39
1:29 | 2:15
-----------
1:27 | 1:38

Ubuntu/Pwsh (24.0)
0:54 | 0:56
0:51 | 0:48
0:48 | 0:49
0:53 | 0:54
0:50 | 0:50
-----------
0:51 | 0:51

pre 24.0 vs. 24.0 doesn't seem to be a big difference In general, pwsh seems to be faster than PowerShell, not significantly. Ubuntu is much faster than Windows.

This is using BcContainerHelper preview - not any new stuff.

@MattTraxinger - if you see big differences between pre 24.0 and 24.0 using CompilerFolder functionality, please create a new issue and include full logs with timestamps for both runs for me to see what the differences are.

Thanks

[RonKoppelaar]

@freddydk which branch do you mean? On github only see master branch and a few older ones. I guess you are refering to your forked branch?

[freddydk]

The branch used in the PR:

This one (from my fork) https://github.com/freddydk/navcontainerhelper/tree/sessions

[freddydk]

It will be in BcContainerHelper preview in approx. an hour

[freddydk]

when this is done https://github.com/microsoft/navcontainerhelper/actions/runs/8781315198 it autodeploys to preview

[marknitek]

@freddydk under pwsh sqlserver module does not work currently and from what i know you removed jit install of this because of #3460. Is there a solution for this currently? Or do we have to decide wheter to use pwsh and not have sqlserver module support or use ps5?

We rely on having at least invoke-sqlcmd from the sqlserver module available in our scripts from inside the container.

[freddydk]

If we can find a good way to install invoke-sqlcmd in pwsh (either as part of the generic image) or as part of New-BcContainer - I am all for adding it again, but I found that it wasn't straightforward, which is why I removed it. I will work on re-adding it, but until then - using ps5 in these cases is the only real solution.

[marknitek]

@freddydk may i ask what the problem about installing it is? I just tried it on my side and was able to install it without any issues under pwsh, so i'am wondering why it may not work in some scenarios?

[freddydk]

Here's what I found when looking at this

• JIT installing doesn't work if the container for some reason cannot do DNS lookups (frequent problem)
• If you install the sqlserver module under pwsh - you will need to modify the parameters of invoke-sqlcmd as it always uses encrypted connections - need a way to be able to use the same invoke-sqlcmd from ps5 and ps7 then I could install this in the generic image.
• If I install the old sqlserver where encryption wasn't needed in the generic image, then the image will get flagged as being not updated with the latest security patches.

I will be looking for a solution for no. 2 - help is welcome:-)

[RonKoppelaar]

Testing the improved BCCOntainerhelper Unfortune don’t see much improvements (yet). My test case is as follow:

• Create (onprem)container
• Get a list of apps (stored in shared folder)
• Perform Get-NavContainerAppInfo using the filepath to gather manifest info

I run the same test for BC23.5 and BC24 from PS ISE

import-module "C:\Projects\Git\navcontainerhelper\BcContainerHelper.psm1" -Force

$Start=(get-date)
write-host "Getting manifest info"
$OnpremContainer="demo"
$AppFolder="C:\ProgramData\BcContainerHelper\Extensions\backup\Apps24"
write-verbose "Reading manifest info of the apps in $AppFolder"
$AppFilenames = (Get-ChildItem -Path "$AppFolder\*.app" ).FullName
$AppFileDict = @{}
Foreach($File in $AppFilenames ) {
    Write-Verbose "Read AppInfo for: $(Split-Path -Path $File -Leaf)"
    $AppInfo = Get-NavContainerAppInfo -appFilePath $file -containerName $OnpremContainer
    $Key = "$($AppInfo.AppId)-$($AppInfo.Version.ToString())"
    $AppFileDict.Add($key,$File)
}
((Get-date)-$Start).TotalSeconds

Processing it on demo container (BC24)  181 secs Processing it on dev23x container (BC23.5)  52.7 secs

@freddydk Will send you the list of apps in a private message...

[freddydk]

@RonKoppelaar when trying your script on a bc23 and a bc24 container - I get 47,9 seconds and 48,1 seconds (with my changes) Note that containers needs to be generated with this version of ContainerHelper as well in order to be able to run winrm sessions.

[RonKoppelaar]

Indeed after -recreating my BC24 container with the new BCContainerhelper I have simular results. 51secs. Great!!! Next step will be to run my pipeline against the preview version of bccontainerhelper. Keep you posted

[freddydk]

@marknitek: This runs in PS5:

This runs in PS7 (after installing the latest SqlServer module):

I have to add -encrypt optional to all invoke-sqlcmd's

Maybe there is a way to set this as a default???

[RonKoppelaar]

Testing with the preview build. Performance seems to be improved. Running into following error on statement:

Backup-BcContainerDatabases -containerName $ContainerName -bakFolder $BackupFolder

Backing up mydatabase to C:\ProgramData\BcContainerHelper\extensions\C186936\Backup\database.bak The term 'Backup-SqlDatabase' is not recognized as a name of a cmdlet, function, script file, or executable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Exception Script Stack Trace: at Backup, : line 20 at , : line 65

No UsePSw24 parameter exists though..

[marknitek]

@freddydk yeah i did the same but added -TrustServerCertificate instead of -Encrypt From what i know there is and will not be any option to set another default value for the -Encrypt Parameter (due to the new security standards).

How about overwriting the Invoke-SqlCmd Command and appending the -Encrypt Parameter? I don't know where we nee to place the overwrite, because it depends if we are in the winrm session or using docker exec.

But this works on my side when using the winrm session (just a quick test that i have done)

function Invoke-SqlCmd {
    SqlServer\Invoke-Sqlcmd @args -Encrypt Optional 
}

[RonKoppelaar]

I workaround the missing SQLServer module like this... Invoke-ScriptInNavContainer -containerName $ContainerName -usePwsh:$false -scriptblock { Install-PackageProvider -Name NuGet -MinimumVersion '2.8.5.201' -Force Install-Module -Name SqlServer -RequiredVersion 21.1.18256 -Force -AllowClobber }

The Install-PackageProvider was needed to bypass an issue with non-interactive PS when invoked from AzureDevOps.

[freddydk]

@RonKoppelaar - but this will install the SqlServer module in PS5 where it isn't needed. I am creating a new insider generic image (automatically selected by the preview of BcContainerHelper now). Version 1.0.2.20 That one has SqlServer PS module installed in PS7 with 3 overrides suggested by @marknitek With this invoke-sqlcmd, backup-sqldatabase and restore-sqldatabase should work under ps7 as well and I will remove the -usepwsh $false for these and test.

[freddydk]

Latest BcContainerHelper preview should have resolved all issues explained in this post. Please test and let's ship this to prod as soon as possible

[RonKoppelaar]

I tested, but bumped into following error when creating an onprem Container with option USeNewDatabase Full log attached. createContainer.log

Attaching files as new Database mytempdb Putting database CRONUS back online Removing Database CRONUS from localhost\SQLEXPRESS Creating new database CRONUS on localhost\SQLEXPRESS with default Collation Could not load type 'Microsoft.Extensions.Logging.ILoggingBuilder' from assembly 'Microsoft.Extensions.Logging.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'.

Exception Script Stack Trace: at , : line 45

PowerShell Call Stack: at Invoke-ScriptInBcContainer, C:\Users\buildadmin2\Documents\WindowsPowerShell\Modules\bccontainerhelper\6.0.16\ContainerHandling\Invoke-ScriptInNavContainer.ps1: line 68 at Clean-BcContainerDatabase, C:\Users\buildadmin2\Documents\WindowsPowerShell\Modules\bccontainerhelper\6.0.16\AppHandling\Clean-BcContainerDatabase.ps1: line 99 at New-BcContainer, C:\Users\buildadmin2\Documents\WindowsPowerShell\Modules\bccontainerhelper\6.0.16\ContainerHandling\New-NavContainer.ps1: line 2442 at New-cdsaBCContainer, C:\Users\buildadmin2\Documents\WindowsPowerShell\Modules\cdsa.build.al\1.0.186918\public\container\New-cdsaBCContainer.ps1: line 136 at , C:\agent2_work\1\s\DevOps.Builds\ERP AL\Invoke-FullBuild-Onprem.ps1: line 55 at , : line 1 at , C:\agent2_work_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\4.238.3\azurepowershell.ps1: line 261 at , C:\agent2_work_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\4.238.3\azurepowershell.ps1: line 257 at , : line 1 at , : line 22 at , : line 18 at , : line 1

Container Free Physical Memory: 45.7Gb