microsoft / navcontainerhelper

Official Microsoft repository for BcContainerHelper, a PowerShell module, which makes it easier to work with Business Central Containers on Docker.
MIT License
379 stars 243 forks source link

Issue with connecting to remote SQL database. #1518

Closed GreatScott000 closed 3 years ago

GreatScott000 commented 3 years ago

PLEASE DO NOT INCLUDE ANY PASSWORDS OR TOKENS IN YOUR ISSUE!!!

Describe the issue I am trying to a database that is on a SQL server that is separate from the Docker host server. I have tried a number of configurations and I am unable to get it work. You provided a suggestion in the past to get it past that worked when the Docker host and SQL server were the same but I cannot get it to work when they are separate. I am using that in the script below. I have tried various changes that include different options for the -dns and using the SQL servername or ip Address.

Any suggestions?

Scripts used to create container and cause the issue

Clear-Host
$ipAddress = '10.0.1.4'

$containerName = 'vect09045'
$password = 'P@ssw0rd'
$securePassword = ConvertTo-SecureString -String $password -AsPlainText -Force
$credential = New-Object pscredential 'admin', $securePassword
$auth = 'UserPassword'
$artifactUrl = Get-NavArtifactUrl -nav '2016' -cu '45' -country 'gb'
$databaseServer = '10.0.1.4'
$databaseInstance = ''
$databaseName = '"vect09045"'
$databaseUsername = 'Docker'
$databasePassword = '6Hash######'
$databaseSecurePassword = ConvertTo-SecureString -String $databasePassword -AsPlainText -Force
$databaseCredential = New-Object pscredential $databaseUsername, $databaseSecurePassword
$licenseFile = 'F:\License\NAV2018License.flf'
New-BcContainer `
    -accept_eula `
    -containerName $containerName `
    -credential $credential `
    -auth $auth `
    -artifactUrl $artifactUrl `
    -databaseServer $databaseServer -databaseInstance $databaseInstance -databaseName $databaseName `
    -databaseCredential $databaseCredential `
    -licenseFile $licenseFile `
    -dns '10.0.1.4' `
    -myScripts @{ "SetupDatabase.ps1" = "Add-Content -Path 'C:\Windows\System32\drivers\etc\hosts' -Value '$ipaddress $databaseServer'; . 'c:\run\SetupDatabase.ps1'" } `
    -updateHosts 

Full output of scripts

BcContainerHelper is version 1.0.15
BcContainerHelper is running as administrator
Host is Microsoft Windows Server 2019 Datacenter - ltsc2019
Docker Client Version is 19.03.12
Docker Server Version is 19.03.12
Removing container vect09045
Removing f:\bccontainerhelper\Extensions\vect09045
Fetching all docker images
Using image mcr.microsoft.com/businesscentral:10.0.17763.1577
Creating Container vect09045
Version: 9.0.51476.0-gb
Style: onprem
Multitenant: No
Platform: 9.0.51476.0
Generic Tag: 1.0.1.0
Container OS Version: 10.0.17763.1577 (ltsc2019)
Host OS Version: 10.0.17763.1577 (ltsc2019)
Using process isolation
Using locale en-GB
Disabling the standard eventlog dump to container log every 2 seconds (use -dumpEventLog to enable)
Using license file F:\License\NAV2018License.flf
Files in f:\bccontainerhelper\Extensions\vect09045\my:
- AdditionalOutput.ps1
- license.flf
- MainLoop.ps1
- SetupDatabase.ps1
- SetupVariables.ps1
- updatecontainerhosts.ps1
Creating container vect09045 from image mcr.microsoft.com/businesscentral:10.0.17763.1577
220761db264caab265da505980d8774cd86397161799c248809408f08ea29917
Waiting for container vect09045 to be ready
Using artifactUrl https://bcartifacts.azureedge.net/onprem/9.0.51476.0/gb
Using installer from C:\Run\90
Copying Platform Artifacts
Copying Application Artifacts
Copying Installers
Copying TestToolKit
Copying UpgradeToolKit
Installing NAV
Starting Local SQL Server
WARNING: Waiting for service 'SQL Server (SQLEXPRESS) (MSSQL$SQLEXPRESS)' to 
start...
Installing Url Rewrite
Installing SQL Clr Types
Installing Report Viewer
Installing OpenXML
Starting Internet Information Server
Copying Service Tier Files
Copying Web Client Files
Copying Windows Client Files
Copying PowerShell Scripts
Copying Test Assemblies
Copying TestToolKit
Copying UpgradeToolKit
Copying ClientUserSettings
Copying ReportBuilder
Skipping restore of Cronus database
Installing C:\NAVDVD\Installers\GB\OlAddin\OutlookAddIn.Local.Gb.msi
Installing C:\NAVDVD\Installers\GB\RTC\RoleTailoredClient.Local.Gb.msi
Installing C:\NAVDVD\Installers\GB\Server\Server.Local.Gb.msi
Installing C:\NAVDVD\Installers\GB\WebClient\WebClient.Local.Gb.msi
Skipping C:\NAVDVD\Installers\GB\WebHelp\WebHelp.Local.Gb.msi
Modifying NAV Service Tier Config File for Docker
Creating NAV Service Tier
Installing SIP crypto provider: 'C:\Windows\System32\NavSip.dll'
Installation took 82 seconds
Installation complete
Initializing...
Setting host.containerhelper.internal to 172.30.144.1 in container hosts file
Starting Container
Hostname is vect09045
PublicDnsName is vect09045
Using NavUserPassword Authentication
Import Encryption Key
Cannot establish a connection to the SQL Server/Database. 

This could be due to one of the following reasons:

* The SQL server is not started.
* The database does not exist.
* Proper permissions have not been given to the NAV Server Account.

Try again later or contact your system administrator.
at <ScriptBlock>, C:\run\SetupDatabase.ps1: line 152
at <ScriptBlock>, C:\Run\my\SetupDatabase.ps1: line 2
at <ScriptBlock>, C:\Run\navstart.ps1: line 121
at <ScriptBlock>, C:\Run\start.ps1: line 350
at <ScriptBlock>, <No file>: line 1Error
Using artifactUrl https://bcartifacts.azureedge.net/onprem/9.0.51476.0/gb
Using installer from C:\Run\90
Copying Platform Artifacts
Copying Application Artifacts
Copying Installers
Copying TestToolKit
Copying UpgradeToolKit
Installing NAV
Starting Local SQL Server
WARNING: Waiting for service 'SQL Server (SQLEXPRESS) (MSSQL$SQLEXPRESS)' to 
start...
Installing Url Rewrite
Installing SQL Clr Types
Installing Report Viewer
Installing OpenXML
Starting Internet Information Server
Copying Service Tier Files
Copying Web Client Files
Copying Windows Client Files
Copying PowerShell Scripts
Copying Test Assemblies
Copying TestToolKit
Copying UpgradeToolKit
Copying ClientUserSettings
Copying ReportBuilder
Skipping restore of Cronus database
Installing C:\NAVDVD\Installers\GB\OlAddin\OutlookAddIn.Local.Gb.msi
Installing C:\NAVDVD\Installers\GB\RTC\RoleTailoredClient.Local.Gb.msi
Installing C:\NAVDVD\Installers\GB\Server\Server.Local.Gb.msi
Installing C:\NAVDVD\Installers\GB\WebClient\WebClient.Local.Gb.msi
Skipping C:\NAVDVD\Installers\GB\WebHelp\WebHelp.Local.Gb.msi
Modifying NAV Service Tier Config File for Docker
Creating NAV Service Tier
Installing SIP crypto provider: 'C:\Windows\System32\NavSip.dll'
Installation took 82 seconds
Installation complete
Initializing...
Setting host.containerhelper.internal to 172.30.144.1 in container hosts file
Starting Container
Hostname is vect09045
PublicDnsName is vect09045
Using NavUserPassword Authentication
Import Encryption Key
Cannot establish a connection to the SQL Server/Database. 

This could be due to one of the following reasons:

* The SQL server is not started.
* The database does not exist.
* Proper permissions have not been given to the NAV Server Account.

Try again later or contact your system administrator.
at <ScriptBlock>, C:\run\SetupDatabase.ps1: line 152
at <ScriptBlock>, C:\Run\my\SetupDatabase.ps1: line 2
at <ScriptBlock>, C:\Run\navstart.ps1: line 121
at <ScriptBlock>, C:\Run\start.ps1: line 350
at <ScriptBlock>, <No file>: line 1
Initialization of container vect09045 failed
At C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\1.0.15\ContainerHandling\Wait-NavContainerReady.ps1:43 char:17
+ ...             throw "Initialization of container $containerName failed" ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Initialization ...ect09045 failed:String) [], RuntimeException
    + FullyQualifiedErrorId : Initialization of container vect09045 failed

...

Screenshots If applicable, add screenshots to help explain your problem.

Additional context

freddydk commented 3 years ago

-dns should point to a dns server. I do not assume that your SQL server is a DNS Server as well? when you enter the container (enter-bccontainer in powershell ise) - can you ping 10.0.1.4?

GreatScott000 commented 3 years ago

The SQL Server and DNS server are the same (it is also the DC but that is a different conversation :)).

Yes I can ping the ip from the container:

Initialization of container vect09045 failed At C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\1.0.15\ContainerHandling\Wait-NavContainerReady.ps1:43 char:17

  • ... throw "Initialization of container $containerName failed" ...
  • 
    + CategoryInfo          : OperationStopped: (Initialization ...ect09045 failed:String) [], RuntimeException
    + FullyQualifiedErrorId : Initialization of container vect09045 failed

PS C:\windows\system32> Enter-BcContainer -containerName vect09045 Welcome to the NAV Container PowerShell prompt

[VECT09045]: PS C:\Run> ping navdev1 Ping request could not find host navdev1. Please check the name and try again.

[VECT09045]: PS C:\Run> ping 10.0.1.4

Pinging 10.0.1.4 with 32 bytes of data: Reply from 10.0.1.4: bytes=32 time=2ms TTL=127 Reply from 10.0.1.4: bytes=32 time<1ms TTL=127 Reply from 10.0.1.4: bytes=32 time<1ms TTL=127 Reply from 10.0.1.4: bytes=32 time<1ms TTL=127

Ping statistics for 10.0.1.4: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 2ms, Average = 0ms

[VECT09045]: PS C:\Run> type C:\Windows\System32\drivers\etc\hosts


# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
#      102.54.94.97     rhino.acme.com          # source server
#       38.25.63.10     x.acme.com              # x client host
# localhost name resolution is handled within DNS itself.
# 127.0.0.1       localhost
# ::1             localhost
172.30.144.1 host.containerhelper.internal
10.0.1.4 10.0.1.4

[VECT09045]: PS C:\Run> 
freddydk commented 3 years ago

If you set the following in the hosts hosts file:

10.0.1.4 dbserver.containerhelper.internal

Then dbserver.containerhelper.internal should be available in the container and you can use this as databaseServer. You can then remove the setupdatabase override and databaseserver should be set to this. You do have mixed auth enabled on the SQL Server (to allow database login) right?

Do you have other containers connecting to this database?

GreatScott000 commented 3 years ago

Yes, SQL is set to mixed authentication. I did have other containers connecting, unfortunately I have removed them recently as they were no longer needed.

I will make the changes suggested and advise.

GreatScott000 commented 3 years ago

Same results: Here is the new script:

Clear-Host

$containerName = 'vect09045'
$password = 'P@ssw0rd'
$securePassword = ConvertTo-SecureString -String $password -AsPlainText -Force
$credential = New-Object pscredential 'admin', $securePassword
$auth = 'UserPassword'
$artifactUrl = Get-NavArtifactUrl -nav '2016' -cu '45' -country 'gb'
$databaseServer = 'dbserver.containerhelper.internal'
$databaseInstance = ''
$databaseName = 'vect09045'
$databaseUsername = 'Docker'
$databasePassword = '6Hash######'
$databaseSecurePassword = ConvertTo-SecureString -String $databasePassword -AsPlainText -Force
$databaseCredential = New-Object pscredential $databaseUsername, $databaseSecurePassword
$licenseFile = 'F:\License\NAV2018License.flf'
New-BcContainer `
    -accept_eula `
    -containerName $containerName `
    -credential $credential `
    -auth $auth `
    -artifactUrl $artifactUrl `
    -databaseServer $databaseServer -databaseInstance $databaseInstance -databaseName $databaseName `
    -databaseCredential $databaseCredential `
    -licenseFile $licenseFile `
    -dns '10.0.1.4' `
    -updateHosts 

#    -myScripts @{ "SetupDatabase.ps1" = "Add-Content -Path 'C:\Windows\System32\drivers\etc\hosts' -Value '$ipaddress $sqlserver'; . 'c:\run\SetupDatabase.ps1'" } `

and the new output

BcContainerHelper is version 1.0.15 BcContainerHelper is running as administrator Host is Microsoft Windows Server 2019 Datacenter - ltsc2019 Docker Client Version is 19.03.12 Docker Server Version is 19.03.12 Removing container vect09045 Removing vect09045 from host hosts file Removing vect09045-* from host hosts file Removing f:\bccontainerhelper\Extensions\vect09045 Fetching all docker images Using image mcr.microsoft.com/businesscentral:10.0.17763.1577 Creating Container vect09045 Version: 9.0.51476.0-gb Style: onprem Multitenant: No Platform: 9.0.51476.0 Generic Tag: 1.0.1.0 Container OS Version: 10.0.17763.1577 (ltsc2019) Host OS Version: 10.0.17763.1577 (ltsc2019) Using process isolation Using locale en-GB Disabling the standard eventlog dump to container log every 2 seconds (use -dumpEventLog to enable) Using license file F:\License\NAV2018License.flf Files in f:\bccontainerhelper\Extensions\vect09045\my:

  • AdditionalOutput.ps1
  • license.flf
  • MainLoop.ps1
  • SetupVariables.ps1
  • updatehosts.ps1 Creating container vect09045 from image mcr.microsoft.com/businesscentral:10.0.17763.1577 cb000a21c5eee86941bdbb02f5f22744ed97eabb7854c3d778adf13565981ac4 Waiting for container vect09045 to be ready Using artifactUrl https://bcartifacts.azureedge.net/onprem/9.0.51476.0/gb Using installer from C:\Run\90 Copying Platform Artifacts Copying Application Artifacts Copying Installers Copying TestToolKit Copying UpgradeToolKit Installing NAV Starting Local SQL Server Installing Url Rewrite Installing SQL Clr Types Installing Report Viewer Installing OpenXML Starting Internet Information Server Copying Service Tier Files Copying Web Client Files Copying Windows Client Files Copying PowerShell Scripts Copying Test Assemblies Copying TestToolKit Copying UpgradeToolKit Copying ClientUserSettings Copying ReportBuilder Skipping restore of Cronus database Installing C:\NAVDVD\Installers\GB\OlAddin\OutlookAddIn.Local.Gb.msi Installing C:\NAVDVD\Installers\GB\RTC\RoleTailoredClient.Local.Gb.msi Installing C:\NAVDVD\Installers\GB\Server\Server.Local.Gb.msi Installing C:\NAVDVD\Installers\GB\WebClient\WebClient.Local.Gb.msi Skipping C:\NAVDVD\Installers\GB\WebHelp\WebHelp.Local.Gb.msi Modifying NAV Service Tier Config File for Docker Creating NAV Service Tier Installing SIP crypto provider: 'C:\Windows\System32\NavSip.dll' Installation took 83 seconds Installation complete Initializing... Setting dbserver.containerhelper.internal to 10.0.1.4 in container hosts file (copy from host hosts file) Setting host.containerhelper.internal to 172.30.144.1 in container hosts file Starting Container Hostname is vect09045 PublicDnsName is vect09045 Using NavUserPassword Authentication Import Encryption Key Cannot establish a connection to the SQL Server/Database.

This could be due to one of the following reasons:

  • The SQL server is not started.
  • The database does not exist.
  • Proper permissions have not been given to the NAV Server Account.

Try again later or contact your system administrator. at , C:\Run\SetupDatabase.ps1: line 152 at , C:\Run\navstart.ps1: line 121 at , C:\Run\start.ps1: line 350 at , : line 1Error Using artifactUrl https://bcartifacts.azureedge.net/onprem/9.0.51476.0/gb Using installer from C:\Run\90 Copying Platform Artifacts Copying Application Artifacts Copying Installers Copying TestToolKit Copying UpgradeToolKit Installing NAV Starting Local SQL Server Installing Url Rewrite Installing SQL Clr Types Installing Report Viewer Installing OpenXML Starting Internet Information Server Copying Service Tier Files Copying Web Client Files Copying Windows Client Files Copying PowerShell Scripts Copying Test Assemblies Copying TestToolKit Copying UpgradeToolKit Copying ClientUserSettings Copying ReportBuilder Skipping restore of Cronus database Installing C:\NAVDVD\Installers\GB\OlAddin\OutlookAddIn.Local.Gb.msi Installing C:\NAVDVD\Installers\GB\RTC\RoleTailoredClient.Local.Gb.msi Installing C:\NAVDVD\Installers\GB\Server\Server.Local.Gb.msi Installing C:\NAVDVD\Installers\GB\WebClient\WebClient.Local.Gb.msi Skipping C:\NAVDVD\Installers\GB\WebHelp\WebHelp.Local.Gb.msi Modifying NAV Service Tier Config File for Docker Creating NAV Service Tier Installing SIP crypto provider: 'C:\Windows\System32\NavSip.dll' Installation took 83 seconds Installation complete Initializing... Setting dbserver.containerhelper.internal to 10.0.1.4 in container hosts file (copy from host hosts file) Setting host.containerhelper.internal to 172.30.144.1 in container hosts file Starting Container Hostname is vect09045 PublicDnsName is vect09045 Using NavUserPassword Authentication Import Encryption Key Cannot establish a connection to the SQL Server/Database.

This could be due to one of the following reasons:

  • The SQL server is not started.
  • The database does not exist.
  • Proper permissions have not been given to the NAV Server Account.

Try again later or contact your system administrator. at , C:\Run\SetupDatabase.ps1: line 152 at , C:\Run\navstart.ps1: line 121 at , C:\Run\start.ps1: line 350 at , : line 1 Initialization of container vect09045 failed At C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\1.0.15\ContainerHandling\Wait-NavContainerReady.ps1:43 char:17

  • ... throw "Initialization of container $containerName failed" ...
  • 
    + CategoryInfo          : OperationStopped: (Initialization ...ect09045 failed:String) [], RuntimeException
    + FullyQualifiedErrorId : Initialization of container vect09045 failed

and this is the host's hosts file:


PS C:\windows\system32> type C:\Windows\System32\drivers\etc\hosts
# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
#      102.54.94.97     rhino.acme.com          # source server
#       38.25.63.10     x.acme.com              # x client host
# localhost name resolution is handled within DNS itself.
#   127.0.0.1       localhost
#   ::1             localhost
172.25.203.123 my
172.29.26.248 vecta
172.29.21.75 vectb
172.18.73.190 wisbtest
172.21.134.141 wisb11001
172.21.219.190 wisb14017
172.30.230.46 wisb16005
172.30.90.25 smal09055
172.30.86.70 smal10014
172.30.91.147 smal11007
172.30.82.87 smal13001
172.30.92.195 smal14008
172.30.89.192 smal15003
172.30.92.219 smal16003
172.31.2.5 mariosapp-ci
172.31.15.116 myapp-ci
172.31.12.159 murisapp-ci
172.31.12.159 murisapp-ci-default
172.31.2.5 mariosapp-ci-default
172.31.15.116 myapp-ci-default
172.31.8.74 fintools-ci
172.31.8.74 fintools-ci-default
172.31.9.224 vect17000
172.30.232.20 test
172.30.226.16 test2
172.23.80.164 nav09045
172.23.89.147 bc14017
172.23.83.109 bc14017upg
172.23.87.78 vect
10.0.1.4 dbserver.containerhelper.internal
10.0.1.4 NAVDEV1
172.19.219.16 vect14017
GreatScott000 commented 3 years ago

So I installed SQL local on the Docker server and the script works. I really don't want that but now I can move forward at least.
I add this to host's hosts file and which is the local server's ip address.

10.0.1.7 dbserver.containerhelper.internal

Clear-Host

$containerName = 'vect09045'
$password = 'P@ssw0rd'
$securePassword = ConvertTo-SecureString -String $password -AsPlainText -Force
$credential = New-Object pscredential 'admin', $securePassword
$auth = 'UserPassword'
$artifactUrl = Get-NavArtifactUrl -nav '2016' -cu '45' -country 'gb'
#$databaseServer = 'dbserver.containerhelper.internal'
$databaseServer = 'dbserver.containerhelper.internal'
$databaseInstance = ''
$databaseName = 'vect09045'
$databaseUsername = 'Docker'
$databasePassword = '6Hash######'
$databaseSecurePassword = ConvertTo-SecureString -String $databasePassword -AsPlainText -Force
$databaseCredential = New-Object pscredential $databaseUsername, $databaseSecurePassword
$licenseFile = 'F:\License\NAV2018License.flf'
New-BcContainer `
    -accept_eula `
    -containerName $containerName `
    -credential $credential `
    -auth $auth `
    -artifactUrl $artifactUrl `
    -databaseServer $databaseServer -databaseInstance $databaseInstance -databaseName $databaseName `
    -databaseCredential $databaseCredential `
    -licenseFile $licenseFile `
    -dns '10.0.1.4' `
    -updateHosts 

#    -myScripts @{ "SetupDatabase.ps1" = "Add-Content -Path 'C:\Windows\System32\drivers\etc\hosts' -Value '$ipaddress $sqlserver'; . 'c:\run\SetupDatabase.ps1'" } `
GreatScott000 commented 3 years ago

Hmmm... I may have found the problem with the remote SQL server.

I realised I still had a container (but not the script) that used the remote SQL server. It happens to be a BC14 so it has the C/Side Client installed. I run it up and get this message: image

The login does not have an expiry date but obviously something is wrong with the account. image

I have recreated the login but still getting the error.

GreatScott000 commented 3 years ago

Nevermind! I was looking at the wrong SQL server in SSMS. There was a password expiration on the account in that server.

Sometimes there are just too many things of which to keep track!

GreatScott000 commented 3 years ago

Fixed he expired login and was able to create the container (after a little hiccough, see #1523).

I wonder if there is anyway you can get more details information about the connection error? A lot of programs have issues with this though.

Thanks again for the assistance.

freddydk commented 3 years ago

I will see if there is an innerexception I can surface.

BTW - when connecting to an external database server from two different containers, you will need to share the same encryption key file. After creating the first container, the key file is in the my folder - you can share that in -myscripts to the second container.

freddydk commented 3 years ago

If using a foreign database, you could generate a DynamicsNAV.key on the host somewhere and share that to -myscripts, encrypted with the password of the databasecredentials for the external database - then all containers should use the same and always work.

GreatScott000 commented 3 years ago

I have been recreating the container so I will figure out how to do that. I can't find it in the container help source.

GreatScott000 commented 3 years ago

So this is what you are talking about... https://docs.microsoft.com/en-us/powershell/module/microsoft.dynamics.nav.management/new-navencryptionkey?view=businesscentral-ps-16#description

freddydk commented 3 years ago

Yes After you create a container with a foreign database connection, you will have a DynamicsNAV.key in the "my" folder of the container. You can copy this to a file share or a secure URL and reuse the same for the same database server connection. It will be encrypted (encryptionpassword) with the databaseCredential.password so it will work for any other containers connecting to that database server.