search

microsoft / navcontainerhelper

Official Microsoft repository for BcContainerHelper, a PowerShell module, which makes it easier to work with Business Central Containers on Docker.
MIT License
385 stars 247 forks source link

vNext container doesn't start properly - Failed to start service 'Dynamics 365 Business Central Server #3300

Closed Koubek closed 9 months ago

Koubek commented 10 months ago

Describe the issue Container fails on the following step: 

Starting Service Tier
Failed to start service 'Dynamics 365 Business Central Server [BC] (MicrosoftDynamicsNavServer$BC)'.

Scripts used to create container and cause the issue We have this problem when running BC24 containers with and also without BcContainerHelper.

Full output of scripts

Installing from artifacts
Starting Local SQL Server
Starting Internet Information Server
Copying Service Tier Files
c:\dl\sandbox\24.0.15506.0\platform\ServiceTier\Program Files
c:\dl\sandbox\24.0.15506.0\platform\ServiceTier\System64Folder
Copying PowerShell Scripts
c:\dl\sandbox\24.0.15506.0\platform\WindowsPowerShellScripts\Cloud\NAVAdministration
c:\dl\sandbox\24.0.15506.0\platform\WindowsPowerShellScripts\WebSearch
Copying Web Client Files
c:\dl\sandbox\24.0.15506.0\platform\WebClient\Microsoft Dynamics NAV
Copying ModernDev Files
c:\dl\sandbox\24.0.15506.0\platform
c:\dl\sandbox\24.0.15506.0\platform\ModernDev\program files\Microsoft Dynamics NAV
Copying additional files
Copying ConfigurationPackages
C:\dl\sandbox\24.0.15506.0\platform\ConfigurationPackages
Copying Test Assemblies
C:\dl\sandbox\24.0.15506.0\platform\Test Assemblies
Copying Extensions
C:\dl\sandbox\24.0.15506.0\w1\Extensions
Copying Applications
C:\dl\sandbox\24.0.15506.0\platform\Applications
Copying dependencies
Copying ReportBuilder
Importing PowerShell Modules
Skipping restore of Cronus database
Modifying Business Central Service Tier Config File for Docker
Creating Business Central Service Tier
Installing SIP crypto provider: 'C:\Windows\System32\NavSip.dll'
Installation took 13 seconds
Installation complete
Initializing...
Welcome to the Business Central Container PowerShell prompt
NP: NAV/BC version number is: 24.0.15490.0
Non-authoritative answer:
NP: Trying to enable Docker Artifact File Share ...
Creating new drive:
New drive should be ready.
NP: Docker Artifact File Share enabled.
NP: Copying Docker Artifact File Share ...
NP: Processing scripts for Docker Artifacts ...
NP: Updating fonts...
An error occured installing 'C:\navipartner\artifacts\fonts\Standard\lucon.ttf'
Access to the path 'C:\Windows\Fonts\lucon.ttf' is denied.
NP: Updating branding...
NP: ***
NP: ***
NP: [#609969 / *** ]: Install dotnet and potentially missing stuff...
Patching container to install dotnet 8.0.0
Downloading c:\run\my\dotnet-win8.exe
Generic image is 1.0.2.14 or below, installing dotnet 8.0.0
Starting Container
Hostname is NP609969
PublicDnsName is NP609969.***
Using NavUserPassword Authentication
NP: Processing SetupDatabase.ps1
Import Encryption Key
NP: Exiting SetupDatabase.ps1
Stopping local SQL Server
Creating Self Signed Certificate
Self Signed Certificate Thumbprint F5F92D200EF1C1B627D21496CF143D64467FDA53
DNS identity NP609969.***
Modifying Service Tier Config File with Instance Specific Settings
NP: Modifying Service Tier Config File with Instance Specific Settings
Setting DataCacheSize to 6
Setting ClientServicesMaxConcurrentConnections to 50
Creating ClientServicesMaxNumberOfOrphanedConnections and setting it to 5
Setting ClientServicesIdleClientTimeout to 05:00:00
Setting ClientServicesOperationTimeout to 05:00:00
Setting ClientServicesMaxUploadSize to 500
Creating ClientServicesReconnectPeriod and setting it to 02:00:00
Setting MaxConcurrentCalls to 10
Setting TaskSchedulerMaximumConcurrentRunningTasks to 1
Setting NavHttpClientMaxResponseContentSize to 500
Setting ApiServicesEnabled to true
NP: Applying users NST appSettings
  ManagementServicesIdleClientTimeout = 06:00:00
Setting ManagementServicesIdleClientTimeout to 06:00:00
  ApplicationInsightsInstrumentationKey = ***
Setting ApplicationInsightsInstrumentationKey to ***
  AzureKeyVaultAppSecretsPublisherValidationEnabled = false
Setting AzureKeyVaultAppSecretsPublisherValidationEnabled to false
  ApiServicesEnabled = true
Setting ApiServicesEnabled to true
NP: Custom SetupConfiguration.ps1 has been successfully finished.
NP: Azure Key Vault settings found, setting it up...
WARNING: The new settings value will not take effect until you stop and restart
 the service.
WARNING: The new settings value will not take effect until you stop and restart
 the service.
WARNING: The new settings value will not take effect until you stop and restart
 the service.
NP: Azure Key Vault setup complete.
Starting Service Tier
Failed to start service 'Dynamics 365 Business Central Server [BC] (MicrosoftDynamicsNavServer$BC)'.

Additional context

freddydk commented 10 months ago

Does the event log in the container reveal why the service tier doesn't start?

Koubek commented 10 months ago

Unfortunately no, it fails and restarts. I will try to keep it running, check again, and return back to you.

Koubek commented 10 months ago 
Failed to start service 'Dynamics 365 Business Central Server [BC] (MicrosoftDynamicsNavServer$BC)'.
at <ScriptBlock>, C:\Run\navstart.ps1: line 172
at <ScriptBlock>, C:\Run\my\navstart.ps1: line 2
at <ScriptBlock>, C:\Run\start.ps1: line 384
at <ScriptBlock>, <No file>: line 1
TimeGenerated : 1/20/2024 9:07:14 AM
EntryType     : Error
Message       : Server instance: BC
                Tenant ID: 
                Environment Name: 
                Environment Type: 
                User: 
                Type: System.InvalidOperationException
                Message: This operation is not supported for a relative URI.
                Source: System.Private.Uri
                HResult: -2146233079
                StackTrace:
                     at System.Uri.get_AbsoluteUri()
                     at Microsoft.Dynamics.Nav.Types.AzureKeyVaultSettings..cto
                r(String authority, Guid clientId, StoreLocation 
                certStoreLocation, StoreName certStoreName, String 
                certThumbprint, Uri keyIdentifierUrl)
                     at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVault
                TokenCredentialFactory.<>c.<.ctor>b__5_0()
                     at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
                     at System.Lazy`1.ExecutionAndPublication(LazyHelper 
                executionAndPublication, Boolean useDefaultConstructor)
                     at System.Lazy`1.CreateValue()
                     at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVault
                TokenCredentialFactory..ctor()
                     at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposur
                ePolicyProvider.KeyVaultDynamicPolicyProvider.CreateAndConfigur
                eAzureKeyVaultFacadeBasedOnServerSettings()
                     at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposur
                ePolicyProvider.KeyVaultDynamicPolicyProvider..ctor()
                     at Microsoft.Dynamics.Nav.Runtime.NavEnvironment..ctor(Nav
                EnvironmentFlags flags)
                     at 
                Microsoft.Dynamics.Nav.Runtime.NavEnvironment.get_Instance()
                     at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindows
                Service.SetNavEnviroment(String 
                commandLineServiceInstanceName, ServerUserSettings settings)
                     at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindows
                Service.Start(String commandLineServiceInstanceName)
                     at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindows
                Service.OnStart(String[] args, ServiceBase serviceBase)
                     at Microsoft.Dynamics.Nav.WindowsServices.WindowsService.N
                avWindowsServiceLifetime.OnStart(String[] args)
                     at System.ServiceProcess.ServiceBase.ServiceQueuedMainCall
                back(Object state)
                  --- End of stack trace from previous location ---
                     at System.ServiceProcess.ServiceBase.Run(ServiceBase[] 
                services)
                     at System.ServiceProcess.ServiceBase.Run(ServiceBase 
                service)
                     at Microsoft.Extensions.Hosting.WindowsServices.WindowsSer
                viceLifetime.Run()
                  --- End of stack trace from previous location ---
                     at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(C
                ancellationToken cancellationToken)
                     at Microsoft.Extensions.Hosting.HostingAbstractionsHostExt
                ensions.RunAsync(IHost host, CancellationToken token)
                     at Microsoft.Extensions.Hosting.HostingAbstractionsHostExt
                ensions.RunAsync(IHost host, CancellationToken token)
                     at Microsoft.Extensions.Hosting.HostingAbstractionsHostExt
                ensions.Run(IHost host)
                     at Microsoft.Dynamics.Nav.WindowsServices.DynamicsNavServe
                r.Main(String[] args)
                StackTrace:
                     at System.Uri.get_AbsoluteUri()
                     at Microsoft.Dynamics.Nav.Types.AzureKeyVaultSettings..cto
                r(String authority, Guid clientId, StoreLocation 
                certStoreLocation, StoreName certStoreName, String 
                certThumbprint, Uri keyIdentifierUrl)
                     at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVault
                TokenCredentialFactory.<>c.<.ctor>b__5_0()
                     at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
                     at System.Lazy`1.ExecutionAndPublication(LazyHelper 
                executionAndPublication, Boolean useDefaultConstructor)
                     at System.Lazy`1.CreateValue()
                     at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVault
                TokenCredentialFactory..ctor()
                     at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposur
                ePolicyProvider.KeyVaultDynamicPolicyProvider.CreateAndConfigur
                eAzureKeyVaultFacadeBasedOnServerSettings()
                     at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposur
                ePolicyProvider.KeyVaultDynamicPolicyProvider..ctor()
                     at Microsoft.Dynamics.Nav.Runtime.NavEnvironment..ctor(Nav
                EnvironmentFlags flags)
                     at 
                Microsoft.Dynamics.Nav.Runtime.NavEnvironment.get_Instance()
                     at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindows
                Service.SetNavEnviroment(String 
                commandLineServiceInstanceName, ServerUserSettings settings)
                     at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindows
                Service.Start(String commandLineServiceInstanceName)
                     at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindows
                Service.OnStart(String[] args, ServiceBase serviceBase)
                     at Microsoft.Dynamics.Nav.WindowsServices.WindowsService.N
                avWindowsServiceLifetime.OnStart(String[] args)
                     at System.ServiceProcess.ServiceBase.ServiceQueuedMainCall
                back(Object state)
                  --- End of stack trace from previous location ---
                     at System.ServiceProcess.ServiceBase.Run(ServiceBase[] 
                services)
                     at System.ServiceProcess.ServiceBase.Run(ServiceBase 
                service)
                     at Microsoft.Extensions.Hosting.WindowsServices.WindowsSer
                viceLifetime.Run()
                  --- End of stack trace from previous location ---
                     at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(C
                ancellationToken cancellationToken)
                     at Microsoft.Extensions.Hosting.HostingAbstractionsHostExt
                ensions.RunAsync(IHost host, CancellationToken token)
                     at Microsoft.Extensions.Hosting.HostingAbstractionsHostExt
                ensions.RunAsync(IHost host, CancellationToken token)
                     at Microsoft.Extensions.Hosting.HostingAbstractionsHostExt
                ensions.Run(IHost host)
                     at Microsoft.Dynamics.Nav.WindowsServices.DynamicsNavServe
                r.Main(String[] args)
Koubek commented 10 months ago

I will try to check how the app in Entra is configured. Maybe there is something wrong but it fails only now and on BC24 preview.

slaven commented 10 months ago

From what I saw in logs, this may be an issue with artifacts (?): 24.0.15468.0 - everything works fine 24.0.15491.0 - issues described by @Koubek

Logs don't really give any hint on why the restart fails, even the stack trace is useless in this case.

Koubek commented 10 months ago

I don't see anything looking odd (when checking the app configuration).

Koubek commented 10 months ago

@slaven, maybe the logs I captured are not useless. Let's see what @freddydk says ;)

Koubek commented 10 months ago

I can see NST parameter AzureKeyVaultKeyUri is empty. But on BC23 is also empty.

freddydk commented 10 months ago

Do you have a script with which I can repro this? (the simplest possible repro please)

When I run 24.0 with New-BcContainer, it works fine (with the countries I have tried) What is different here?

freddydk commented 10 months ago

This:

$auth = "UserPassword"
$containerName = "bcserver"
$credential = New-Object pscredential -ArgumentList 'admin', $PasswordSecret.SecretValue
$artifactUrl = Get-BCArtifactUrl -country us -select NextMajor -accept_insiderEula
New-BCContainer -containerName $containerName `
                -accept_eula -accept_insiderEula `
                -Auth $auth `
                -artifactUrl $artifactUrl `
                -Credential $credential `
                -dns "hostDNS"

Works fine - I would need to know what you do differently if I am to help.

Koubek commented 9 months ago

We set up, as mentioned before, Azure Key Vault. And it doesn't work only in that case. AKV integration is probably the root problem and it started to fail quite recently on BC24 (vNext).

I pasted log entries I captured previously: https://github.com/microsoft/navcontainerhelper/issues/3300#issuecomment-1901936792

Here is the important part indicating it might be an issue related to AKV integration. Unfortunately, with AKV I can't share settings as these are very critical bits of info that nobody would like to share :)

at <ScriptBlock>, C:\Run\navstart.ps1: line 172
at <ScriptBlock>, C:\Run\my\navstart.ps1: line 2
at <ScriptBlock>, C:\Run\start.ps1: line 384
at <ScriptBlock>, <No file>: line 1
TimeGenerated : 1/20/2024 9:07:14 AM
EntryType     : Error
Message       : Server instance: BC
                Tenant ID: 
                Environment Name: 
                Environment Type: 
                User: 
                Type: System.InvalidOperationException
                Message: This operation is not supported for a relative URI.
                Source: System.Private.Uri
                HResult: -2146233079
                StackTrace:
                     at System.Uri.get_AbsoluteUri()
                     at Microsoft.Dynamics.Nav.Types.AzureKeyVaultSettings..cto
                r(String authority, Guid clientId, StoreLocation 
                certStoreLocation, StoreName certStoreName, String 
                certThumbprint, Uri keyIdentifierUrl)
                     at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVault
                TokenCredentialFactory.<>c.<.ctor>b__5_0()
Koubek commented 9 months ago

Regarding the previous, maybe it's better to say explicitly - the latest BC24 works fine WITHOUT Azure Key Vault enabled since we install dotnet 8 (without this version of dotnet it wasn't working even without AKV but this was solved previously).

So you have to enable AKV to be able to reproduce the error.

freddydk commented 9 months ago

Sounds like a problem with your Entra App Registration, maybe the Application ID Uri is wrong or some other Uri setting - I would look there? I don't think this is a ContainerHelper or a docker problem though. If you still think this is a ContainerHelper or docker problem, feel free to create another (temporary) Key Vault with a dummy secret and an Entra App ID, which repros the problem and share the secrets with me in email or here. (I could do the same, but if the problem indeed is in setting up the entra app, I might not repro the problem)

Koubek commented 9 months ago

Thanks Freddy.

I know it's unrelated, the problem is when I posted the last time an issue in nav-docker repository, I didn't get a response during a very long time. It was responded actually on Yammer because I posted the question there as nothing was happening on GitHub. And the I closed the issue in GitHub.

I can see there are other questions without any response now so the right repo seems to be dead meanwhile the wrapper repo (BcContainerHelper) is still getting responses. That's why I asked here.

Regarding Entra App, I didn't see anything wrong but I will check again and I will go through the documentation. What is weird is the fact it stopped working now and only for vNext. This means there were some changes also made on your side probably and to understand where the problem is it would be great to know something about those changes.

freddydk commented 9 months ago

Yeah, if a problem is docker related, then this repo is fine - I just don't think that this problem is related to docker or the ContainerHelper. A few releases ago we changed/obsoleted some stuff on the AAD App registration - maybe 24.0 removed support for the old way of doing things - I don't know.

So, nothing weird in things stops working in vNext - but in generel - if I need to look at something, I am very fast if I get a repro - it saves everybody a ton of time.

So, create a new keyvault, a new AAD app and a PowerShell scripts that repros the problem - then you might have a resolution within hours or days.

Complaining about that we don't describe changes and/or we don't reply to issues doesn't bring us closer to a resolution to the problem. Help me help you by creating a repro (that also ensures that the problem actually is with me:-))

freddydk commented 9 months ago

@Koubek did you resolve the issue? Was the problem indeed in the Application ID Uri?

nicolassaleron commented 9 months ago

Same new issue as @Koubek when using Set-BcContainerKeyVaultAadAppAndCertificate.

I will investigate next week as we have also changed our domain certificate until the last execution...

2024-01-26T17:13:48.7021698Z Initialization took 26 seconds
2024-01-26T17:13:48.7023857Z Ready for connections!
2024-01-26T17:13:49.8202280Z Reading CustomSettings.config from bc-latest-70
2024-01-26T17:13:50.1113873Z Creating Desktop Shortcuts for bc-latest-70
2024-01-26T17:13:50.2954002Z Cleanup old dotnet core assemblies
2024-01-26T17:13:51.9667976Z Container bc-latest-70 successfully created
2024-01-26T17:13:52.1159471Z 
2024-01-26T17:13:52.1168196Z Use:
2024-01-26T17:13:52.1183967Z Get-BcContainerEventLog -containerName bc-latest-70 to retrieve a snapshot of the event log from the container
2024-01-26T17:13:52.1199384Z Get-BcContainerDebugInfo -containerName bc-latest-70 to get debug information about the container
2024-01-26T17:13:52.1214072Z Enter-BcContainer -containerName bc-latest-70 to open a PowerShell prompt inside the container
2024-01-26T17:13:52.1228256Z Remove-BcContainer -containerName bc-latest-70 to remove the container again
2024-01-26T17:13:52.1242265Z docker logs bc-latest-70 to retrieve information about URL's again
2024-01-26T17:13:53.2897515Z Downloading certificate file to container
2024-01-26T17:13:53.2931979Z Downloading using WebClient
2024-01-26T17:13:54.1722592Z Keyvault Certificate Thumbprint: D******A
2024-01-26T17:13:54.9017338Z Restarting Service Tier
2024-01-26T17:13:57.5537362Z ServerInstance 'MicrosoftDynamicsNavServer$BC' failed to reach  status 'Running' on this server due to the following error: 'Cannot start the service.'. Additional information may be in the event log of the server.
2024-01-26T17:13:57.5541547Z 
2024-01-26T17:13:57.5551331Z Exception Script Stack Trace:
2024-01-26T17:13:57.5568421Z at <ScriptBlock>, <No file>: line 26
2024-01-26T17:13:57.5572743Z 
2024-01-26T17:13:57.5581227Z PowerShell Call Stack:
2024-01-26T17:13:57.5591970Z at Invoke-ScriptInBcContainer, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.0\ContainerHandling\Invoke-ScriptInNavContainer.ps1: line 71
2024-01-26T17:13:57.5598305Z at Set-BcContainerKeyVaultAadAppAndCertificate, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.0\ContainerHandling\Set-BcContainerKeyVaultAadAppAndCertificate.ps1: line 57
2024-01-26T17:13:57.5604433Z at <ScriptBlock>, C:\agent\_work\2\s\Powershell\NewBCDevelopmentContainer@1.ps1: line 101
2024-01-26T17:13:57.5610654Z at <ScriptBlock>, C:\agent\_work\_temp\9694e79a-3ef6-41eb-9968-a2d913dcbcaa.ps1: line 4
2024-01-26T17:13:57.5616647Z at <ScriptBlock>, <No file>: line 1
2024-01-26T17:13:57.9498986Z 
2024-01-26T17:13:57.9502455Z Container Free Physical Memory: 4.8Gb
2024-01-26T17:13:57.9505644Z 
2024-01-26T17:13:57.9508018Z Services in container bc-latest-70:
2024-01-26T17:13:57.9703585Z - MicrosoftDynamicsNavServer$BC is Stopped
2024-01-26T17:13:57.9722928Z - MSSQL$SQLEXPRESS is Running
2024-01-26T17:13:57.9729641Z 
2024-01-26T17:13:57.9868358Z 
2024-01-26T17:13:57.9872164Z Relevant event log from container bc-latest-70:
2024-01-26T17:13:57.9908808Z - 20240126 05:13:56 - MicrosoftDynamicsNavServer$BC
2024-01-26T17:13:57.9939352Z 
2024-01-26T17:13:57.9939798Z   Server instance: BC
2024-01-26T17:13:57.9940117Z   Tenant ID: 
2024-01-26T17:13:57.9940431Z   Environment Name: 
2024-01-26T17:13:57.9941004Z   Environment Type: 
2024-01-26T17:13:57.9941335Z   User: 
2024-01-26T17:13:57.9941674Z   Type: System.InvalidOperationException
2024-01-26T17:13:57.9942122Z   Message: This operation is not supported for a relative URI.
2024-01-26T17:13:57.9942558Z   Source: System.Private.Uri
2024-01-26T17:13:57.9942876Z   HResult: -2146233079
2024-01-26T17:13:57.9943177Z   StackTrace:
2024-01-26T17:13:57.9943517Z        at System.Uri.get_AbsoluteUri()
2024-01-26T17:13:57.9944217Z        at Microsoft.Dynamics.Nav.Types.AzureKeyVaultSettings..ctor(String authority, Guid clientId, StoreLocation certStoreLocation, StoreName certStoreName, String certThumbprint, Uri keyIdentifierUrl)
2024-01-26T17:13:57.9945031Z        at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVaultTokenCredentialFactory.<>c.<.ctor>b__5_0()
2024-01-26T17:13:57.9945368Z        at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
2024-01-26T17:13:57.9945935Z        at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
2024-01-26T17:13:57.9946239Z        at System.Lazy`1.CreateValue()
2024-01-26T17:13:57.9946499Z        at System.Lazy`1.get_Value()
2024-01-26T17:13:57.9946784Z        at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVaultTokenCredentialFactory..ctor()
2024-01-26T17:13:57.9947422Z        at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposurePolicyProvider.KeyVaultDynamicPolicyProvider.CreateAndConfigureAzureKeyVaultFacadeBasedOnServerSettings()
2024-01-26T17:13:57.9947896Z        at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposurePolicyProvider.KeyVaultDynamicPolicyProvider..ctor()
2024-01-26T17:13:57.9948270Z        at Microsoft.Dynamics.Nav.Runtime.NavEnvironment..ctor(NavEnvironmentFlags flags)
2024-01-26T17:13:57.9948587Z        at Microsoft.Dynamics.Nav.Runtime.NavEnvironment.get_Instance()
2024-01-26T17:13:57.9948980Z        at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.SetNavEnviroment(String commandLineServiceInstanceName, ServerUserSettings settings)
2024-01-26T17:13:57.9949436Z        at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.Start(String commandLineServiceInstanceName)
2024-01-26T17:13:57.9949838Z        at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.OnStart(String[] args, ServiceBase serviceBase)
2024-01-26T17:13:57.9950230Z        at Microsoft.Dynamics.Nav.WindowsServices.WindowsService.NavWindowsServiceLifetime.OnStart(String[] args)
2024-01-26T17:13:57.9950586Z        at System.ServiceProcess.ServiceBase.ServiceQueuedMainCallback(Object state)
2024-01-26T17:13:57.9950982Z     --- End of stack trace from previous location ---
2024-01-26T17:13:57.9951257Z        at System.ServiceProcess.ServiceBase.Run(ServiceBase[] services)
2024-01-26T17:13:57.9951548Z        at System.ServiceProcess.ServiceBase.Run(ServiceBase service)
2024-01-26T17:13:57.9951856Z        at Microsoft.Extensions.Hosting.WindowsServices.WindowsServiceLifetime.Run()
2024-01-26T17:13:57.9952148Z     --- End of stack trace from previous location ---
2024-01-26T17:13:57.9952454Z        at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
2024-01-26T17:13:57.9952835Z        at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
2024-01-26T17:13:57.9953234Z        at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
2024-01-26T17:13:57.9953597Z        at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
2024-01-26T17:13:57.9953931Z        at Microsoft.Dynamics.Nav.WindowsServices.DynamicsNavServer.Main(String[] args)
2024-01-26T17:13:57.9954175Z   StackTrace:
2024-01-26T17:13:57.9954377Z        at System.Uri.get_AbsoluteUri()
2024-01-26T17:13:57.9954789Z        at Microsoft.Dynamics.Nav.Types.AzureKeyVaultSettings..ctor(String authority, Guid clientId, StoreLocation certStoreLocation, StoreName certStoreName, String certThumbprint, Uri keyIdentifierUrl)
2024-01-26T17:13:57.9955491Z        at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVaultTokenCredentialFactory.<>c.<.ctor>b__5_0()
2024-01-26T17:13:57.9955834Z        at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
2024-01-26T17:13:57.9956167Z        at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
2024-01-26T17:13:57.9956531Z        at System.Lazy`1.CreateValue()
2024-01-26T17:13:57.9956876Z        at System.Lazy`1.get_Value()
2024-01-26T17:13:57.9957197Z        at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVaultTokenCredentialFactory..ctor()
2024-01-26T17:13:57.9957886Z        at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposurePolicyProvider.KeyVaultDynamicPolicyProvider.CreateAndConfigureAzureKeyVaultFacadeBasedOnServerSettings()
2024-01-26T17:13:57.9958543Z        at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposurePolicyProvider.KeyVaultDynamicPolicyProvider..ctor()
2024-01-26T17:13:57.9959145Z        at Microsoft.Dynamics.Nav.Runtime.NavEnvironment..ctor(NavEnvironmentFlags flags)
2024-01-26T17:13:57.9959492Z        at Microsoft.Dynamics.Nav.Runtime.NavEnvironment.get_Instance()
2024-01-26T17:13:57.9960112Z        at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.SetNavEnviroment(String commandLineServiceInstanceName, ServerUserSettings settings)
2024-01-26T17:13:57.9960841Z        at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.Start(String commandLineServiceInstanceName)
2024-01-26T17:13:57.9961370Z        at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.OnStart(String[] args, ServiceBase serviceBase)
2024-01-26T17:13:57.9961835Z        at Microsoft.Dynamics.Nav.WindowsServices.WindowsService.NavWindowsServiceLifetime.OnStart(String[] args)
2024-01-26T17:13:57.9962448Z        at System.ServiceProcess.ServiceBase.ServiceQueuedMainCallback(Object state)
2024-01-26T17:13:57.9962825Z     --- End of stack trace from previous location ---
2024-01-26T17:13:57.9963100Z        at System.ServiceProcess.ServiceBase.Run(ServiceBase[] services)
2024-01-26T17:13:57.9963392Z        at System.ServiceProcess.ServiceBase.Run(ServiceBase service)
2024-01-26T17:13:57.9963699Z        at Microsoft.Extensions.Hosting.WindowsServices.WindowsServiceLifetime.Run()
2024-01-26T17:13:57.9963994Z     --- End of stack trace from previous location ---
2024-01-26T17:13:57.9964301Z        at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
2024-01-26T17:13:57.9964775Z        at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
2024-01-26T17:13:57.9965301Z        at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
2024-01-26T17:13:57.9965774Z        at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
2024-01-26T17:13:57.9966761Z        at Microsoft.Dynamics.Nav.WindowsServices.DynamicsNavServer.Main(String[] args)
2024-01-26T17:13:57.9967279Z   
2024-01-26T17:13:57.9967379Z 
2024-01-26T17:13:57.9967657Z - 20240126 05:13:56 - MicrosoftDynamicsNavServer$BC
2024-01-26T17:13:57.9967798Z 
2024-01-26T17:13:57.9967959Z   Server instance: BC
2024-01-26T17:13:57.9968147Z   Category: Runtime
2024-01-26T17:13:57.9968376Z   ClientSessionId: 00000000-0000-0000-0000-000000000000
2024-01-26T17:13:57.9968644Z   ClientActivityId: 00000000-0000-0000-0000-000000000000
2024-01-26T17:13:57.9968917Z   ServerSessionUniqueId: 00000000-0000-0000-0000-000000000000
2024-01-26T17:13:57.9969205Z   ServerActivityId: 00000000-0000-0000-0000-000000000000
2024-01-26T17:13:57.9969444Z   EventTime: 01/26/2024 17:13:56
2024-01-26T17:13:57.9969716Z   Message (InvalidOperationException): RootException: InvalidOperationException
2024-01-26T17:13:57.9970003Z   This operation is not supported for a relative URI.
2024-01-26T17:13:57.9970223Z   ExceptionStackTrace:
2024-01-26T17:13:57.9970434Z      at System.Uri.get_AbsoluteUri()
2024-01-26T17:13:57.9970846Z      at Microsoft.Dynamics.Nav.Types.AzureKeyVaultSettings..ctor(String authority, Guid clientId, StoreLocation certStoreLocation, StoreName certStoreName, String certThumbprint, Uri keyIdentifierUrl)
2024-01-26T17:13:57.9971485Z      at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVaultTokenCredentialFactory.<>c.<.ctor>b__5_0()
2024-01-26T17:13:57.9972011Z      at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
2024-01-26T17:13:57.9972351Z      at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
2024-01-26T17:13:57.9972657Z      at System.Lazy`1.CreateValue()
2024-01-26T17:13:57.9972868Z      at System.Lazy`1.get_Value()
2024-01-26T17:13:57.9973152Z      at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVaultTokenCredentialFactory..ctor()
2024-01-26T17:13:57.9973648Z      at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposurePolicyProvider.KeyVaultDynamicPolicyProvider.CreateAndConfigureAzureKeyVaultFacadeBasedOnServerSettings()
2024-01-26T17:13:57.9974274Z      at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposurePolicyProvider.KeyVaultDynamicPolicyProvider..ctor()
2024-01-26T17:13:57.9974648Z      at Microsoft.Dynamics.Nav.Runtime.NavEnvironment..ctor(NavEnvironmentFlags flags)
2024-01-26T17:13:57.9974962Z      at Microsoft.Dynamics.Nav.Runtime.NavEnvironment.get_Instance()
2024-01-26T17:13:57.9975379Z      at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.SetNavEnviroment(String commandLineServiceInstanceName, ServerUserSettings settings)
2024-01-26T17:13:57.9975824Z      at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.Start(String commandLineServiceInstanceName)
2024-01-26T17:13:57.9976363Z      at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.OnStart(String[] args, ServiceBase serviceBase)
2024-01-26T17:13:57.9976981Z      at Microsoft.Dynamics.Nav.WindowsServices.WindowsService.NavWindowsServiceLifetime.OnStart(String[] args)
2024-01-26T17:13:57.9977331Z      at System.ServiceProcess.ServiceBase.ServiceQueuedMainCallback(Object state)
2024-01-26T17:13:57.9977620Z   --- End of stack trace from previous location ---
2024-01-26T17:13:57.9977892Z      at System.ServiceProcess.ServiceBase.Run(ServiceBase[] services)
2024-01-26T17:13:57.9978184Z      at System.ServiceProcess.ServiceBase.Run(ServiceBase service)
2024-01-26T17:13:57.9978486Z      at Microsoft.Extensions.Hosting.WindowsServices.WindowsServiceLifetime.Run()
2024-01-26T17:13:57.9978770Z   --- End of stack trace from previous location ---
2024-01-26T17:13:57.9979147Z      at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
2024-01-26T17:13:57.9979523Z      at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
2024-01-26T17:13:57.9979924Z      at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
2024-01-26T17:13:57.9980291Z      at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
2024-01-26T17:13:57.9980620Z      at Microsoft.Dynamics.Nav.WindowsServices.DynamicsNavServer.Main(String[] args)
2024-01-26T17:13:57.9980870Z   CallerStackTrace:
2024-01-26T17:13:57.9981131Z      at Microsoft.Dynamics.Nav.WindowsServices.DynamicsNavServer.Main(String[] args)
2024-01-26T17:13:57.9981358Z   
2024-01-26T17:13:57.9981499Z   
2024-01-26T17:13:57.9981657Z   ProcessId: 3244
2024-01-26T17:13:57.9981834Z   Tag: 00008NE
2024-01-26T17:13:57.9982008Z   ThreadId: 1
2024-01-26T17:13:57.9982189Z   CounterInformation: 
2024-01-26T17:13:57.9982377Z   CustomParameters: {
2024-01-26T17:13:57.9982545Z   }
2024-01-26T17:13:57.9982737Z   GatewayCorrelationId: 
2024-01-26T17:13:57.9982906Z 
2024-01-26T17:13:57.9983381Z - 20240126 05:13:56 - MicrosoftDynamicsNavServer$BC
2024-01-26T17:13:57.9983621Z 
2024-01-26T17:13:57.9983887Z   Service cannot be started. System.InvalidOperationException: This operation is not supported for a relative URI.
2024-01-26T17:13:57.9984185Z      at System.Uri.get_AbsoluteUri()
2024-01-26T17:13:57.9984591Z      at Microsoft.Dynamics.Nav.Types.AzureKeyVaultSettings..ctor(String authority, Guid clientId, StoreLocation certStoreLocation, StoreName certStoreName, String certThumbprint, Uri keyIdentifierUrl)
2024-01-26T17:13:57.9985241Z      at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVaultTokenCredentialFactory.<>c.<.ctor>b__5_0()
2024-01-26T17:13:57.9985574Z      at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
2024-01-26T17:13:57.9985913Z      at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
2024-01-26T17:13:57.9986224Z      at System.Lazy`1.CreateValue()
2024-01-26T17:13:57.9986439Z      at System.Lazy`1.get_Value()
2024-01-26T17:13:57.9986721Z      at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVaultTokenCredentialFactory..ctor()
2024-01-26T17:13:57.9987652Z      at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposurePolicyProvider.KeyVaultDynamicPolicyProvider.CreateAndConfigureAzureKeyVaultFacadeBasedOnServerSettings()
2024-01-26T17:13:57.9988114Z      at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposurePolicyProvider.KeyVaultDynamicPolicyProvider..ctor()
2024-01-26T17:13:57.9988477Z      at Microsoft.Dynamics.Nav.Runtime.NavEnvironment..ctor(NavEnvironmentFlags flags)
2024-01-26T17:13:57.9988790Z      at Microsoft.Dynamics.Nav.Runtime.NavEnvironment.get_Instance()
2024-01-26T17:13:57.9989178Z      at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.SetNavEnviroment(String commandLineServiceInstanceName, ServerUserSettings settings)
2024-01-26T17:13:57.9989617Z      at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.Start(String commandLineServiceInstanceName)
2024-01-26T17:13:57.9990011Z      at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.OnStart(String[] args, ServiceBase serviceBase)
2024-01-26T17:13:57.9990399Z      at Microsoft.Dynamics.Nav.WindowsServices.WindowsService.NavWindowsServiceLifetime.OnStart(String[] args)
2024-01-26T17:13:57.9990748Z      at System.ServiceProcess.ServiceBase.ServiceQueuedMainCallback(Object state)
2024-01-26T17:13:57.9990925Z 
2024-01-26T17:13:57.9995406Z - 20240126 05:13:56 - MicrosoftDynamicsNavServer$BC
2024-01-26T17:13:58.0000223Z 
2024-01-26T17:13:58.0000804Z   Server instance: BC
2024-01-26T17:13:58.0001155Z   Tenant: 
2024-01-26T17:13:58.0001504Z   Environment Name: 
2024-01-26T17:13:58.0001887Z   Environment Type: 
2024-01-26T17:13:58.0003231Z   The service MicrosoftDynamicsNavServer$BC failed to start. This could be caused by a configuration error. Detailed error information: System.InvalidOperationException: This operation is not supported for a relative URI.
2024-01-26T17:13:58.0004044Z      at System.Uri.get_AbsoluteUri()
2024-01-26T17:13:58.0004814Z      at Microsoft.Dynamics.Nav.Types.AzureKeyVaultSettings..ctor(String authority, Guid clientId, StoreLocation certStoreLocation, StoreName certStoreName, String certThumbprint, Uri keyIdentifierUrl)
2024-01-26T17:13:58.0005700Z      at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVaultTokenCredentialFactory.<>c.<.ctor>b__5_0()
2024-01-26T17:13:58.0006302Z      at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
2024-01-26T17:13:58.0006932Z      at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
2024-01-26T17:13:58.0007537Z      at System.Lazy`1.CreateValue()
2024-01-26T17:13:58.0008002Z      at System.Lazy`1.get_Value()
2024-01-26T17:13:58.0008561Z      at Microsoft.Dynamics.Nav.Runtime.AppSecrets.AzureKeyVaultTokenCredentialFactory..ctor()
2024-01-26T17:13:58.0009379Z      at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposurePolicyProvider.KeyVaultDynamicPolicyProvider.CreateAndConfigureAzureKeyVaultFacadeBasedOnServerSettings()
2024-01-26T17:13:58.0010257Z      at Microsoft.Dynamics.Nav.Runtime.Debugger.ResourceExposurePolicyProvider.KeyVaultDynamicPolicyProvider..ctor()
2024-01-26T17:13:58.0010925Z      at Microsoft.Dynamics.Nav.Runtime.NavEnvironment..ctor(NavEnvironmentFlags flags)
2024-01-26T17:13:58.0011533Z      at Microsoft.Dynamics.Nav.Runtime.NavEnvironment.get_Instance()
2024-01-26T17:13:58.0012301Z      at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.SetNavEnviroment(String commandLineServiceInstanceName, ServerUserSettings settings)
2024-01-26T17:13:58.0013140Z      at Microsoft.Dynamics.Nav.WindowsServices.NavServerWindowsService.Start(String commandLineServiceInstanceName).
2024-01-26T17:13:58.0013569Z 
2024-01-26T17:13:58.6784386Z Set-BcContainerKeyVaultAadAppAndCertificate Telemetry Correlation Id: 294ce659-c07e-45fb-9ad8-9e18fe26c803
2024-01-26T17:13:58.7609017Z ServerInstance 'MicrosoftDynamicsNavServer$BC' failed to reach  status 'Running' on this server due to the following 
2024-01-26T17:13:58.7610345Z error: 'Cannot start the service.'. Additional information may be in the event log of the server.
2024-01-26T17:13:58.7610921Z At C:\Program 
2024-01-26T17:13:58.7611611Z Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.0\ContainerHandling\Invoke-ScriptInNavContainer.ps1:112 char:13
2024-01-26T17:13:58.7612020Z +             throw $errorMessage
2024-01-26T17:13:58.7612261Z +             ~~~~~~~~~~~~~~~~~~~
2024-01-26T17:13:58.7612677Z     + CategoryInfo          : OperationStopped: (ServerInstance ... of the server.:String) [], RuntimeException
2024-01-26T17:13:58.7613071Z     + FullyQualifiedErrorId : ServerInstance 'MicrosoftDynamicsNavServer$BC' failed to reach  status 'Running' on this 
2024-01-26T17:13:58.7613488Z     server due to the following error: 'Cannot start the service.'. Additional information may be in the event log of  
2024-01-26T17:13:58.7613768Z    the server.
2024-01-26T17:13:58.7613946Z  
2024-01-26T17:13:58.8452150Z ##[error]PowerShell exited with code '1'.
2024-01-26T17:13:58.8762827Z ##[section]Finishing: Create container
Koubek commented 9 months ago

@Koubek did you resolve the issue? Was the problem indeed in the Application ID Uri?

No, not yet. There are other priorities compared to possible breaking changes in vNext. Sorry!

freddydk commented 9 months ago

FYI, Issue 891 in the AL-Go repo seams to indicate that the problem also happens in the latest 23 version.

Koubek commented 9 months ago

Adding a proper reference to the issues mentioned above: microsoft/AL-Go#891 and also, I have to ask - I can understand Frank didn't change anything in their Entra app at all (at least it's not mentioned there) and BC was or wasn't starting with AKV depending on artifact version.

If this is true, what does it mean for vNext (BC24 preview)??? I still can't run containers with AKV integration, even on 24.0.15733.0/w1 (the latest w1 artifacts available currently).

freddydk commented 9 months ago

Sorry, but I have no idea - I think it is clear by now that the problem is unrelated to BcContainerHelper and docker. I can still investigate if somebody will send me a repro.

freddydk commented 9 months ago

I will investigate which changes where made to 23.3 and see what's up and down.

freddydk commented 9 months ago

Should get fixed in 24 as well soon.

Koubek commented 9 months ago

With vNext I can't check as there are no new vNext artifacts like we had before (w1, dk etc.). There are just platform and core artifacts. It's been like that for several last days. Is it okay like this? Or am I wrong and missing something? Just to clarify, the last one is 24.0.15733.0.

carlosyuwono commented 9 months ago

I encounter the same error when importing the key vault certificate on BC 23.3 or newer. I ran the same script and used the same certificate on BC 23.0 without encountering any errors

Set-BcContainerKeyVaultAadAppAndCertificate -containerName $containerName -pfxFile $pfxFile -pfxPassword $securePfxPassword -clientId $clientId

`BcContainerHelper version 6.0.5-preview1126 BC.HelperFunctions emits usage statistics telemetry to Microsoft Downloading certificate file to container Downloading using WebClient Keyvault Certificate Thumbprint: FB615700DE13078EDB1C0827087B40B7F0927F76 Restarting Service Tier ServerInstance 'MicrosoftDynamicsNavServer$BC' failed to reach status 'Running' on this server due to the following error: 'Cannot start the service.'. Additional information may be in the event log of the server.

Exception Script Stack Trace: at , : line 26

PowerShell Call Stack: at Invoke-ScriptInBcContainer, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.5\ContainerHandling\Invoke-ScriptInNavContainer.ps1: line 71 at Set-BcContainerKeyVaultAadAppAndCertificate, C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.5\ContainerHandling\Set-BcContainerKeyVaultAadAppAndCertificate.ps1: line 57 at , : line 20

Container Free Physical Memory: 29.8Gb

Services in container usn:

Relevant event log from container usn:

Set-BcContainerKeyVaultAadAppAndCertificate Telemetry Correlation Id: 5dceff02-c3f0-41bd-b503-5fb6fcf47cae ServerInstance 'MicrosoftDynamicsNavServer$BC' failed to reach status 'Running' on this server due to the following error: 'Cannot start the service.'. Additional information may be in the event log of the server. At C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\6.0.5\ContainerHandling\Invoke-ScriptInNavContainer.ps1:112 char:13

freddydk commented 9 months ago

It looks like insiders are currently not being deployed due to some problem. We will have to wait until they get these things sorted out.

freddydk commented 9 months ago

NextMajors are again available. You will need BcContainerHelper preview in order to import the test toolkit (as the old version didn't include the business foundation test library). Same inside the generic image - will be updated with the next generic image.

freddydk commented 9 months ago

@Koubek - FYI BC24 will also require PS7.4.1 See https://github.com/microsoft/navcontainerhelper/pull/3341

and again - once new dependencies have settled, a new version of the generic image will be published.

freddydk commented 8 months ago

Generic image with tag 1.0.2.15 has been released