search

bitwarden / server

Bitwarden infrastructure/backend (API, database, Docker, etc).
https://bitwarden.com
Other
15.25k stars 1.27k forks source link

Bitwarden on-premise throws internal server error. #1283

Closed n10u53 closed 3 years ago

n10u53 commented 3 years ago

Hello I am trying to get on-premise hosting working and bitwarden throws an Bad Gateway error. Even when curling on the machine hosting the bitwarden instance. So this has nothing to do with my own world-facing proxy.

"docker ps" shows all things working, even bitwarden-mssql, except for -admin and bitwarden-sso. They both refer to some database error. The bitwarden-mssql shows some errors, but no obvious hard-errors? I am not knowledgable with mssql so could be wrong there... Any advice would be welcomed!

I have tried executing "./bitwarden.sh updatedb" in the hope this would fix anything/show some usefull fails but it says the update and migration was succesfull?? I must be completely overlooking something but I have don't even have the slightest clue what that could be...

bitwarden-sso log:

info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished in 58364.5543ms 500
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 GET http://localhost:5000/alive
fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
      An unhandled exception has occurred while executing the request.
System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server.
The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections
. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server)
   at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential cre
dential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOp
tions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken)
   at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderIn
fo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions option
s, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConne
ction)
   at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal old
Connection)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate,
 Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionPool.WaitForPendingOpen()
--- End of stack trace from previous location where exception was thrown ---
   at Dapper.SqlMapper.QueryAsync[T](IDbConnection cnn, Type effectiveType, CommandDefinition command) in /_/Dapper/SqlMapper.Async.cs:line 419
   at Bit.Core.Repositories.SqlServer.SsoConfigRepository.GetManyByRevisionNotBeforeDate(Nullable`1 notBefore) in /home/runner/work/server/server/src/
Core/Repositories/SqlServer/SsoConfigRepository.cs:line 60
   at Bit.Core.Business.Sso.DynamicAuthenticationSchemeProvider.LoadAllDynamicSchemesIntoCacheAsync() in /home/runner/work/server/server/bitwarden_lic
ense/src/Sso/Utilities/DynamicAuthenticationSchemeProvider.cs:line 161
   at Bit.Core.Business.Sso.DynamicAuthenticationSchemeProvider.GetRequestHandlerSchemesAsync() in /home/runner/work/server/server/bitwarden_license/s
rc/Sso/Utilities/DynamicAuthenticationSchemeProvider.cs:line 141
   at Bit.Sso.Utilities.SsoAuthenticationMiddleware.Invoke(HttpContext context) in /home/runner/work/server/server/bitwarden_license/src/Sso/Utilities
/SsoAuthenticationMiddleware.cs:line 34
   at IdentityServer4.Hosting.BaseUrlMiddleware.Invoke(HttpContext context)
   at Bit.Core.Utilities.CurrentContextMiddleware.Invoke(HttpContext httpContext, ICurrentContext currentContext, GlobalSettings globalSettings) in /h
ome/runner/work/server/server/src/Core/Utilities/CurrentContextMiddleware.cs:line 21
   at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, T
ask task)
ClientConnectionId:00000000-0000-0000-0000-000000000000
fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[3]
      An exception was thrown attempting to execute the error handler.
System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server.
The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections
. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server)
   at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential cre
dential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOp
tions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken)
   at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderIn
fo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions option
s, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConne
ction)
   at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal old
Connection)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate,
 Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionPool.WaitForPendingOpen()
--- End of stack trace from previous location where exception was thrown ---
   at Dapper.SqlMapper.QueryAsync[T](IDbConnection cnn, Type effectiveType, CommandDefinition command) in /_/Dapper/SqlMapper.Async.cs:line 419
   at Bit.Core.Repositories.SqlServer.SsoConfigRepository.GetManyByRevisionNotBeforeDate(Nullable`1 notBefore) in /home/runner/work/server/server/src/
Core/Repositories/SqlServer/SsoConfigRepository.cs:line 60
   at Bit.Core.Business.Sso.DynamicAuthenticationSchemeProvider.LoadAllDynamicSchemesIntoCacheAsync() in /home/runner/work/server/server/bitwarden_lic
ense/src/Sso/Utilities/DynamicAuthenticationSchemeProvider.cs:line 161
   at Bit.Core.Business.Sso.DynamicAuthenticationSchemeProvider.GetRequestHandlerSchemesAsync() in /home/runner/work/server/server/bitwarden_license/s
rc/Sso/Utilities/DynamicAuthenticationSchemeProvider.cs:line 141
   at Bit.Sso.Utilities.SsoAuthenticationMiddleware.Invoke(HttpContext context) in /home/runner/work/server/server/bitwarden_license/src/Sso/Utilities
/SsoAuthenticationMiddleware.cs:line 34
   at IdentityServer4.Hosting.BaseUrlMiddleware.Invoke(HttpContext context)
   at Bit.Core.Utilities.CurrentContextMiddleware.Invoke(HttpContext httpContext, ICurrentContext currentContext, GlobalSettings globalSettings) in /h
ome/runner/work/server/server/src/Core/Utilities/CurrentContextMiddleware.cs:line 21
   at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.HandleException(HttpContext context, ExceptionDispatchInfo edi)
ClientConnectionId:00000000-0000-0000-0000-000000000000
fail: Microsoft.AspNetCore.Server.Kestrel[13]
      Connection id "0HM854KR6LOUJ", Request id "0HM854KR6LOUJ:00000001": An unhandled exception was thrown by the application.
System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server.
The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections
. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server)
   at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential cre
dential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOp
tions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken)
   at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderIn
fo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions option
s, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConne
ction)
   at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal old
Connection)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate,
 Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionPool.WaitForPendingOpen()
--- End of stack trace from previous location where exception was thrown ---
   at Dapper.SqlMapper.QueryAsync[T](IDbConnection cnn, Type effectiveType, CommandDefinition command) in /_/Dapper/SqlMapper.Async.cs:line 419
   at Bit.Core.Repositories.SqlServer.SsoConfigRepository.GetManyByRevisionNotBeforeDate(Nullable`1 notBefore) in /home/runner/work/server/server/src/
Core/Repositories/SqlServer/SsoConfigRepository.cs:line 60
   at Bit.Core.Business.Sso.DynamicAuthenticationSchemeProvider.LoadAllDynamicSchemesIntoCacheAsync() in /home/runner/work/server/server/bitwarden_lic
ense/src/Sso/Utilities/DynamicAuthenticationSchemeProvider.cs:line 161
   at Bit.Core.Business.Sso.DynamicAuthenticationSchemeProvider.GetRequestHandlerSchemesAsync() in /home/runner/work/server/server/bitwarden_license/s
rc/Sso/Utilities/DynamicAuthenticationSchemeProvider.cs:line 141
   at Bit.Sso.Utilities.SsoAuthenticationMiddleware.Invoke(HttpContext context) in /home/runner/work/server/server/bitwarden_license/src/Sso/Utilities
/SsoAuthenticationMiddleware.cs:line 34
   at IdentityServer4.Hosting.BaseUrlMiddleware.Invoke(HttpContext context)
   at Bit.Core.Utilities.CurrentContextMiddleware.Invoke(HttpContext httpContext, ICurrentContext currentContext, GlobalSettings globalSettings) in /h
ome/runner/work/server/server/src/Core/Utilities/CurrentContextMiddleware.cs:line 21
   at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, T
ask task)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.HandleException(HttpContext context, ExceptionDispatchInfo edi)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, T
ask task)
   at Microsoft.AspNetCore.Builder.Extensions.UsePathBaseMiddleware.Invoke(HttpContext context)
   at Bit.Sso.Startup.<>c__DisplayClass9_1.<<Configure>b__2>d.MoveNext() in /home/runner/work/server/server/bitwarden_license/src/Sso/Startup.cs:line
100
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
ClientConnectionId:00000000-0000-0000-0000-000000000000

bitwarden-mssql log:

info: Bit.Migrator.DbMigrator[12482444]
      Migrating database.
fail: Bit.Admin.HostedServices.DatabaseMigrationHostedService[0]
      Database unavailable for migration. Trying again (attempt #2)...
System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server.
The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections
. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server)
   at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential cre
dential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOp
tions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken)
   at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderIn
fo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions option
s, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConne
ction)
   at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal old
Connection)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate,
 Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOpti
ons, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions u
serOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, Tas
kCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletio
nSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.Open()
   at Bit.Migrator.DbMigrator.MigrateMsSqlDatabase(Boolean enableLogging, CancellationToken cancellationToken) in /home/runner/work/server/server/util
/Migrator/DbMigrator.cs:line 41
   at Bit.Admin.HostedServices.DatabaseMigrationHostedService.StartAsync(CancellationToken cancellationToken) in /home/runner/work/server/server/src/A
dmin/HostedServices/DatabaseMigrationHostedService.cs:line 40
ClientConnectionId:00000000-0000-0000-0000-000000000000
info: Bit.Migrator.DbMigrator[12482444]
      Migrating database.
info: Bit.Core.Jobs.JobListener[12482444]
      Starting job DeleteSendsJob at 04/22/2021 00:35:00.
fail: Bit.Admin.HostedServices.DatabaseMigrationHostedService[0]
      Database unavailable for migration. Trying again (attempt #3)...
System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server.
The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections
. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server)
   at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential cre
dential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOp
tions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken)
   at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderIn
fo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions option
s, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConne
ction)
   at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal old
Connection)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate,
 Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOpti
ons, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions u
serOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, Tas
kCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletio
nSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.Open()
   at Bit.Migrator.DbMigrator.MigrateMsSqlDatabase(Boolean enableLogging, CancellationToken cancellationToken) in /home/runner/work/server/server/util
/Migrator/DbMigrator.cs:line 41
   at Bit.Admin.HostedServices.DatabaseMigrationHostedService.StartAsync(CancellationToken cancellationToken) in /home/runner/work/server/server/src/A
dmin/HostedServices/DatabaseMigrationHostedService.cs:line 40
ClientConnectionId:00000000-0000-0000-0000-000000000000

bitwarden-mssql log:

2021-04-22 00:25:36.95 Server      The licensing PID was successfully processed. The new edition is [Express Edition].
2021-04-22 00:25:37.72 Server      Microsoft SQL Server 2017 (RTM-CU18) (KB4527377) - 14.0.3257.3 (X64)
        Nov 16 2019 01:14:50
        Copyright (C) 2017 Microsoft Corporation
        Express Edition (64-bit) on Linux (Ubuntu 16.04.6 LTS)
2021-04-22 00:25:37.73 Server      UTC adjustment: 0:00
2021-04-22 00:25:37.73 Server      (c) Microsoft Corporation.
2021-04-22 00:25:37.73 Server      All rights reserved.
2021-04-22 00:25:37.73 Server      Server process ID is 32.
2021-04-22 00:25:37.73 Server      Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2021-04-22 00:25:37.73 Server      Registry startup parameters:
         -d /var/opt/mssql/data/master.mdf
         -l /var/opt/mssql/data/mastlog.ldf
         -e /var/opt/mssql/log/errorlog
2021-04-22 00:25:37.80 Server      SQL Server detected 1 sockets with 4 cores per socket and 4 logical processors per socket, 4 total logical processo
rs; using 4 logical processors based on SQL Server licensing. This is an informational message; no user action is required.
2021-04-22 00:25:37.81 Server      SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is requi
red.
2021-04-22 00:25:37.81 Server      Detected 12556 MB of RAM. This is an informational message; no user action is required.
2021-04-22 00:25:37.81 Server      Using conventional memory in the memory manager.
2021-04-22 00:25:39.54 Server      Buffer pool extension is already disabled. No action is necessary.
2021-04-22 00:25:40.78 Server      InitializeExternalUserGroupSid failed. Implied authentication will be disabled.
2021-04-22 00:25:40.78 Server      Implied authentication manager initialization failed. Implied authentication will be disabled.
2021-04-22 00:25:40.79 Server      Successfully initialized the TLS configuration. Allowed TLS protocol versions are ['1.0 1.1 1.2']. Allowed TLS ciph
ers are ['ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA25
6:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:A
ES128-SHA:!DHE-RSA-AES256-GCM-SHA384:!DHE-RSA-AES128-GCM-SHA256:!DHE-RSA-AES256-SHA:!DHE-RSA-AES128-SHA'].
2021-04-22 00:25:40.99 Server      The maximum number of dedicated administrator connections for this instance is '1'
2021-04-22 00:25:40.99 Server      Node configuration: node 0: CPU mask: 0x000000000000000f:0 Active CPU mask: 0x000000000000000f:0. This message prov
ides a description of the NUMA configuration for this computer. This is an informational message only. No user action is required.
2021-04-22 00:25:40.99 Server      Using dynamic lock allocation.  Initial allocation of 2500 Lock blocks and 5000 Lock Owner blocks per node.  This i
s an informational message only.  No user action is required.
2021-04-22 00:25:41.00 Server      In-Memory OLTP initialized on lowend machine.
2021-04-22 00:25:41.27 Server      Database Instant File Initialization: enabled. For security and performance considerations see the topic 'Database
Instant File Initialization' in SQL Server Books Online. This is an informational message only. No user action is required.
ForceFlush is enabled for this instance.
2021-04-22 00:25:41.28 Server      Query Store settings initialized with enabled = 1,
2021-04-22 00:25:41.28 Server      Software Usage Metrics is disabled.
2021-04-22 00:25:41.28 spid4s      Starting up database 'master'.
ForceFlush feature is enabled for log durability.
2021-04-22 00:25:44.21 spid4s      Service Master Key could not be decrypted using one of its encryptions. See sys.key_encryptions for details.
2021-04-22 00:25:44.21 spid4s      An error occurred during Service Master Key initialization. SQLErrorCode=33095, State=8, LastOsError=0.
2021-04-22 00:25:44.56 spid4s      SQL Server Audit is starting the audits. This is an informational message. No user action is required.
2021-04-22 00:25:44.68 spid4s      SQL Server Audit has started the audits. This is an informational message. No user action is required.
2021-04-22 00:25:45.55 spid4s      SQL Trace ID 1 was started by login "sa".
2021-04-22 00:25:46.25 spid4s      Server name is 'fe37cfdd9f6b'. This is an informational message only. No user action is required.
2021-04-22 00:25:46.26 spid20s     Always On: The availability replica manager is starting. This is an informational message only. No user action is r
equired.
2021-04-22 00:25:46.26 spid20s     Always On: The availability replica manager is waiting for the instance of SQL Server to allow client connections.
This is an informational message only. No user action is required.
2021-04-22 00:25:46.30 spid10s     Starting up database 'mssqlsystemresource'.
2021-04-22 00:25:46.30 spid4s      Starting up database 'msdb'.
2021-04-22 00:25:46.31 spid10s     The resource database build version is 14.00.3257. This is an informational message only. No user action is require
d.
2021-04-22 00:25:46.46 spid10s     Starting up database 'model'.
2021-04-22 00:25:47.62 spid19s     A self-generated certificate was successfully loaded for encryption.
2021-04-22 00:25:47.62 spid19s     Server is listening on [ 'any' <ipv6> 1433].
2021-04-22 00:25:47.62 spid19s     Server is listening on [ 'any' <ipv4> 1433].
2021-04-22 00:25:47.63 spid19s     Dedicated administrator connection support was not started because it is disabled on this edition of SQL Server. If
 you want to use a dedicated administrator connection, restart SQL Server using the trace flag 7806. This is an informational message only. No user ac
tion is required.
2021-04-22 00:25:47.63 spid19s     SQL Server is now ready for client connections. This is an informational message; no user action is required.
2021-04-22 00:25:48.75 spid10s     Polybase feature disabled.
2021-04-22 00:25:48.75 spid10s     Clearing tempdb database.
2021-04-22 00:25:52.20 spid10s     Starting up database 'tempdb'.
2021-04-22 00:25:54.31 spid20s     The Service Broker endpoint is in disabled or stopped state.
2021-04-22 00:25:54.31 spid20s     The Database Mirroring endpoint is in disabled or stopped state.
2021-04-22 00:25:55.11 spid20s     Service Broker manager has started.
2021-04-22 00:25:55.46 spid4s      Recovery is complete. This is an informational message only. No user action is required.
n10u53 commented 3 years ago

The problem that withheld me from accessing the instance was a nginx error, 502 duuhh... Folowing these steps here fixed it, not exactly sure why, new to docker and pretty late now. So I'll be checking that out tomorrow!

But that still doesn't explain why sso and admin are unhealthy, and why they are throwing these errors...

robbevl commented 3 years ago

I had the exact same issue yesterday. My issue had to do with Docker networking not playing nice with firewalld with nftables backend. Are you on Centos / RHEL / Fedora by any chance?

The msql container was not reachable, causing SSO and Admin containers to fail. The web container as well was not reachable, causing nginx to throw 502s. Both containers did not have web access either. ICMP does work, but no tcp.

What worked for me was to change the firewalld backend to iptables (in /etc/firewalld/firewalld.conf, then restart firewalld, then restart docker).

Google "firewalld nftables docker" for more info ;)

n10u53 commented 3 years ago

I had the exact same issue yesterday. My issue had to do with Docker networking not playing nice with firewalld with nftables backend. Are you on Centos / RHEL / Fedora by any chance?

The msql container was not reachable, causing SSO and Admin containers to fail. The web container as well was not reachable, causing nginx to throw 502s. Both containers did not have web access either. ICMP does work, but no tcp.

What worked for me was to change the firewalld backend to iptables (in /etc/firewalld/firewalld.conf, then restart firewalld, then restart docker).

Google "firewalld nftables docker" for more info ;)

This might very well be my problem yes, I am using Centos. I will check this out and update the awnser with the outcome! Thanks alot! :D

n10u53 commented 3 years ago

I had the exact same issue yesterday. My issue had to do with Docker networking not playing nice with firewalld with nftables backend. Are you on Centos / RHEL / Fedora by any chance?

The msql container was not reachable, causing SSO and Admin containers to fail. The web container as well was not reachable, causing nginx to throw 502s. Both containers did not have web access either. ICMP does work, but no tcp.

What worked for me was to change the firewalld backend to iptables (in /etc/firewalld/firewalld.conf, then restart firewalld, then restart docker).

Google "firewalld nftables docker" for more info ;)

This did indeed fix my issue. I googled around for some other solutions regarding this issue but none of them worked. Your's did right out of the box though! Thanks! Any insights on why the one does and the other doesn't work or what make the difference between the two?

robbevl commented 3 years ago

I'm afraid I'm not knowledgeable on how iptables-nftables work together and how Docker handles its firewalling to explain in detail. But after struggling with a few similar issues like this in the past few days, I'm learning :)

How I understand it, if firewalld has the nftables as backend, the iptables command still works to push changes, but they don't reflect the single source of truth anymore. Docker doesn't know about nftables, and pushes its rules through iptables (which should work), but it also uses the iptables command to see which rules are in place. And seeing there are rules in place iptables -L or -S doesn't know about some things will go wrong. Using iptables as backend seems like the easy solution for now (other than handling your lowlevel firewall rules manually).

Disclaimer: not an expert here, so if anyone feels the need to correct, please do ;)