search

btcpayserver / btcpayserver-docker

Docker resources for hosting BTCPayServer easily
MIT License
560 stars 345 forks source link

Docker Containers starting but no connection possible. LND waiting for Bitcoin. NBExplorer throwing errors. #864

Closed gobrrrme closed 6 months ago

gobrrrme commented 6 months ago

Hi,

I'm running my BTCpayServer with Lightning Node on a RockPro64 with the Hack0 image by Nicolas Dorier.

What happened:

Came home, lightning node was down, restartet the node, everything was fine again. Then I used btcpay-down.sh, and in preparation for an update, I updated the system via apt update && apt upgrade.

The system installed loads of updates and also some language packages (not sure if relevant).

After using btcpay-update.sh, the update script was successful but I got a 502 bad gateway when trying to connect via local IP, and a host error when trying via the cloudflare tunneled clearnet url.

Logs throw lots of errors:

NBexplorer docker logs --tail 100 generated_nbxplorer_1 info: Configuration: Data Directory: /datadir info: Configuration: Configuration File: /datadir/Main/settings.config info: Configuration: Network: Mainnet info: Configuration: Supported chains: BTC info: Configuration: DBCache: 50 MB info: Configuration: Network: Mainnet info: Configuration: Supported chains: BTC info: Configuration: DBCache: 50 MB info: NBXplorer.DatabaseSetup: Postgres services activated Unhandled exception. Npgsql.NpgsqlException (0x80004005): Failed to connect to 172.18.0.9:5432 ---> System.Net.Sockets.SocketException (111): Connection refused at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.gWaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at Npgsql.TaskTimeoutAndCancellation.ExecuteAsync(Func`2 getTaskFunc, NpgsqlTimeout timeout, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.ConnectAsync(NpgsqlTimeout timeout, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.ConnectAsync(NpgsqlTimeout timeout, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.RawOpen(SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken, Boolean isFirstAttempt) at Npgsql.Internal.NpgsqlConnector.g__OpenCore|213_1(NpgsqlConnector conn, SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken, Boolean isFirstAttempt) at Npgsql.Internal.NpgsqlConnector.Open(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.PoolingDataSource.OpenNewConnector(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.PoolingDataSource.gRentAsync|34_0(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.NpgsqlConnection.g__OpenAsync|42_0(Boolean async, CancellationToken cancellationToken) at NBXplorer.Extensions.ReliableOpenConnectionAsync(NpgsqlDataSource ds, CancellationToken cancellationToken) in /source/NBXplorer/Extensions.cs:line 54 at NBXplorer.HostedServices.DatabaseSetupHostedService.StartAsync(CancellationToken cancellationToken) in /source/NBXplorer/HostedServices/DatabaseSetupHostedService.cs:line 36 at NBXplorer.HostedServices.DatabaseSetupHostedService.StartAsync(CancellationToken cancellationToken) in /source/NBXplorer/HostedServices/DatabaseSetupHostedService.cs:line 37 at Microsoft.AspNetCore.Hosting.HostedServiceExecutor.ExecuteAsync(Func`2 callback, Boolean throwOnFirstFailure) at Microsoft.AspNetCore.Hosting.WebHost.StartAsync(CancellationToken cancellationToken) at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage) at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage) at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token) at Microsoft.AspNetCore.Hosting.WebHostExtensions.Run(IWebHost host) at NBXplorer.Program.Main(String[] args) in /source/NBXplorer/Program.cs:line 49

LND
docker logs --tail 100 btcpayserver_lnd_bitcoin alias=Go Brrr | LND added to /data/lnd.conf LND_CHAIN=btc LND_ENVIRONMENT=mainnet Added bitcoin.active and bitcoin.mainnet to config file /data/lnd.conf Waiting /root/.nbxplorer/btc_fully_synched to be created...

Bitcoind docker logs --tail 100 btcpayserver_bitcoind 2023-12-26T08:51:37Z Config file arg: [main] rpcbind="0.0.0.0:43782" 2023-12-26T08:51:37Z Config file arg: [main] rpcport="43782" 2023-12-26T08:51:37Z Config file arg: [main] walletdir="/walletdata/mainnet" 2023-12-26T08:51:37Z Config file arg: [main] whitelist="0.0.0.0/0" 2023-12-26T08:51:37Z Config file arg: [main] zmqpubhashblock="tcp://0.0.0.0:28334" 2023-12-26T08:51:37Z Config file arg: [main] zmqpubrawblock="tcp://0.0.0.0:28332" 2023-12-26T08:51:37Z Config file arg: [main] zmqpubrawtx="tcp://0.0.0.0:28333" 2023-12-26T08:51:37Z Using at most 125 automatic connections (1048576 file descriptors available) 2023-12-26T08:51:37Z Using 16 MiB out of 16 MiB requested for signature cache, able to store 524288 elements 2023-12-26T08:51:37Z Using 16 MiB out of 16 MiB requested for script execution cache, able to store 524288 elements 2023-12-26T08:51:37Z Script verification uses 5 additional threads 2023-12-26T08:51:37Z scheduler thread start 2023-12-26T08:51:37Z Binding RPC on address 0.0.0.0 port 43782 2023-12-26T08:51:37Z WARNING: the RPC server is not safe to expose to untrusted networks such as the public internet 2023-12-26T08:51:37Z [http] creating work queue of depth 16 2023-12-26T08:51:37Z Using random cookie authentication. 2023-12-26T08:51:37Z Generated RPC authentication cookie /home/bitcoin/.bitcoin/.cookie 2023-12-26T08:51:37Z Using rpcauth authentication. 2023-12-26T08:51:37Z [http] starting 4 worker threads 2023-12-26T08:51:37Z Using wallet directory /walletdata/mainnet 2023-12-26T08:51:37Z init message: Verifying wallet(s)… 2023-12-26T08:51:37Z Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010) 2023-12-26T08:51:37Z Using wallet /walletdata/mainnet/wallet.dat 2023-12-26T08:51:37Z BerkeleyEnvironment::Open: LogDir=/walletdata/mainnet/database ErrorFile=/walletdata/mainnet/db.log 2023-12-26T08:51:38Z Using /16 prefix for IP bucketing 2023-12-26T08:51:38Z init message: Loading P2P addresses… 2023-12-26T08:51:38Z Loaded 74827 addresses from peers.dat 727ms 2023-12-26T08:51:38Z init message: Loading banlist… 2023-12-26T08:51:38Z SetNetworkActive: true 2023-12-26T08:51:39Z Cache configuration: 2023-12-26T08:51:39Z Using 2.0 MiB for block index database 2023-12-26T08:51:39Z Using 8.0 MiB for chain state database 2023-12-26T08:51:39Z * Using 440.0 MiB for in-memory UTXO set (plus up to 476.8 MiB of unused mempool space) 2023-12-26T08:51:39Z init message: Loading block index… 2023-12-26T08:51:39Z Assuming ancestors of block 00000000000000000001a0a448d6cf2546b06801389cc030b2b18c6491266815 have valid signatures. 2023-12-26T08:51:39Z Setting nMinimumChainWork=000000000000000000000000000000000000000052b2559353df4117b7348b64 2023-12-26T08:51:39Z Prune configured to target 50000 MiB on disk for block and undo files. 2023-12-26T08:51:39Z Opening LevelDB in /home/bitcoin/.bitcoin/blocks/index 2023-12-26T08:51:39Z Opened LevelDB successfully 2023-12-26T08:51:39Z Using obfuscation key for /home/bitcoin/.bitcoin/blocks/index: 0000000000000000 2023-12-26T08:51:49Z LoadBlockIndexDB: last block file = 4018 2023-12-26T08:51:49Z LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=13, size=23387832, heights=822994...823006, time=2023-12-26...2023-12-26) 2023-12-26T08:51:49Z Checking all blk files are present... 2023-12-26T08:51:49Z LoadBlockIndexDB(): Block files have previously been pruned 2023-12-26T08:51:52Z Initializing chainstate Chainstate [ibd] @ height -1 (null) 2023-12-26T08:51:52Z Opening LevelDB in /home/bitcoin/.bitcoin/chainstate 2023-12-26T08:51:52Z Opened LevelDB successfully 2023-12-26T08:51:52Z Using obfuscation key for /home/bitcoin/.bitcoin/chainstate: 116f3a8903bf13bc 2023-12-26T08:51:53Z Loaded best chain: hashBestChain=00000000000000000002617bb7f8c7c995d007078ef3aac168ece37f75af1b0e height=823006 date=2023-12-26T08:42:48Z progress=0.999997 2023-12-26T08:51:53Z Opening LevelDB in /home/bitcoin/.bitcoin/chainstate 2023-12-26T08:51:53Z Opened LevelDB successfully 2023-12-26T08:51:53Z Using obfuscation key for /home/bitcoin/.bitcoin/chainstate: 116f3a8903bf13bc 2023-12-26T08:51:53Z [Chainstate [ibd] @ height 823006 (00000000000000000002617bb7f8c7c995d007078ef3aac168ece37f75af1b0e)] resized coinsdb cache to 8.0 MiB 2023-12-26T08:51:53Z [Chainstate [ibd] @ height 823006 (00000000000000000002617bb7f8c7c995d007078ef3aac168ece37f75af1b0e)] resized coinstip cache to 440.0 MiB 2023-12-26T08:51:53Z init message: Verifying blocks… 2023-12-26T08:51:53Z Verifying last 6 blocks at level 3 2023-12-26T08:51:53Z Verification progress: 0% 2023-12-26T08:51:56Z Verification progress: 16% 2023-12-26T08:51:58Z Verification progress: 33% 2023-12-26T08:51:59Z Verification progress: 50% 2023-12-26T08:52:01Z Verification progress: 66% 2023-12-26T08:52:02Z Verification progress: 83% 2023-12-26T08:52:03Z Verification progress: 99% 2023-12-26T08:52:03Z Verification: No coin database inconsistencies in last 6 blocks (25735 transactions) 2023-12-26T08:52:03Z block index 24921ms 2023-12-26T08:52:03Z init message: Loading wallet… 2023-12-26T08:52:16Z BerkeleyEnvironment::Open: LogDir=/walletdata/mainnet/database ErrorFile=/walletdata/mainnet/db.log 2023-12-26T08:52:17Z [default wallet] Wallet file version = 10500, last client version = 260000 2023-12-26T08:52:19Z [default wallet] Legacy Wallet Keys: 2268 plaintext, 0 encrypted, 2268 w/ metadata, 2268 total. 2023-12-26T08:52:19Z [default wallet] Descriptors: 0, Descriptor Keys: 0 plaintext, 0 encrypted, 0 total. 2023-12-26T08:52:19Z [default wallet] Wallet completed loading in 2297ms 2023-12-26T08:52:19Z [default wallet] setKeyPool.size() = 2000 2023-12-26T08:52:19Z [default wallet] mapWallet.size() = 9 2023-12-26T08:52:19Z [default wallet] m_address_book.size() = 801 2023-12-26T08:52:19Z init message: Pruning blockstore… 2023-12-26T08:52:19Z block tree size = 823038 2023-12-26T08:52:19Z nBestHeight = 823006 2023-12-26T08:52:19Z initload thread start 2023-12-26T08:52:19Z torcontrol thread start 2023-12-26T08:52:19Z Bound to 127.0.0.1:8334 2023-12-26T08:52:19Z Bound to [::]:39388 2023-12-26T08:52:19Z Bound to 0.0.0.0:39388 2023-12-26T08:52:19Z Loaded 0 addresses from "anchors.dat" 2023-12-26T08:52:19Z 0 block-relay-only anchors will be tried for connections. 2023-12-26T08:52:19Z init message: Starting network threads… 2023-12-26T08:52:19Z net thread start 2023-12-26T08:52:19Z dnsseed thread start 2023-12-26T08:52:19Z addcon thread start 2023-12-26T08:52:19Z init message: Done loading 2023-12-26T08:52:19Z msghand thread start 2023-12-26T08:52:19Z opencon thread start 2023-12-26T08:52:19Z Waiting 300 seconds before querying DNS seeds. 2023-12-26T08:52:20Z Leaving InitialBlockDownload (latching to false) 2023-12-26T08:52:47Z New outbound-full-relay v1 peer connected: version: 70015, blocks=823005, peer=0 2023-12-26T08:52:55Z New outbound-full-relay v1 peer connected: version: 70016, blocks=823006, peer=1 2023-12-26T08:53:03Z P2P peers available. Skipped DNS seeding. 2023-12-26T08:53:03Z dnsseed thread exit 2023-12-26T08:55:58Z New outbound-full-relay v1 peer connected: version: 70016, blocks=823006, peer=2 2023-12-26T08:56:20Z New outbound-full-relay v1 peer connected: version: 70015, blocks=823006, peer=3 2023-12-26T08:57:34Z New outbound-full-relay v1 peer connected: version: 70015, blocks=823006, peer=7`

BTCPAY docker logs --tail 100 generated_btcpayserver_1 at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(Boolean errorsExpected, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenAsync(CancellationToken cancellationToken, Boolean errorsExpected) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.Exists(Boolean async, CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.Exists(Boolean async, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Migrations.HistoryRepository.ExistsAsync(CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Migrations.HistoryRepository.GetAppliedMigrationsAsync(CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Migrations.Internal.NpgsqlMigrator.MigrateAsync(String targetMigration, CancellationToken cancellationToken) at BTCPayServer.Hosting.MigrationStartupTask.Migrate(CancellationToken cancellationToken) in /source/BTCPayServer/Hosting/MigrationStartupTask.cs:line 862 at BTCPayServer.Hosting.MigrationStartupTask.ExecuteAsync(CancellationToken cancellationToken) in /source/BTCPayServer/Hosting/MigrationStartupTask.cs:line 85 at Microsoft.AspNetCore.Hosting.WebHostExtensions.StartWithTasksAsync(IWebHost webHost, CancellationToken cancellationToken) in /source/BTCPayServer/Extensions/WebHostExtensions.cs:line 19 at BTCPayServer.Program.Main(String[] args) in /source/BTCPayServer/Program.cs:line 76 at BTCPayServer.Program.Main(String[] args) in /source/BTCPayServer/Program.cs:line 103 at BTCPayServer.Program.<Main>(String[] args) bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) info: Configuration: Data Directory: /datadir info: Configuration: Configuration File: /datadir/Main/settings.config info: BTCPayServer.Plugins.PluginManager: Loading plugins from /root/.btcpayserver/Plugins info: BTCPayServer.Plugins.PluginManager: Adding and executing plugin BTCPayServer - 1.12.3 info: BTCPayServer.Plugins.PluginManager: Adding and executing plugin BTCPayServer.Plugins.Shopify - 1.12.3 info: BTCPayServer.Plugins.PluginManager: Adding and executing plugin BTCPayServer.Plugins.PointOfSale - 1.12.3 info: BTCPayServer.Plugins.PluginManager: Adding and executing plugin BTCPayServer.Plugins.PayButton - 1.12.3 info: BTCPayServer.Plugins.PluginManager: Adding and executing plugin BTCPayServer.Plugins.NFC - 1.12.3 info: BTCPayServer.Plugins.PluginManager: Adding and executing plugin BTCPayServer.Plugins.Crowdfund - 1.12.3 info: BTCPayServer.Plugins.PluginManager: Adding and executing plugin BTCPayServer.Plugins.Bitcoin - 1.12.3 info: BTCPayServer.Plugins.PluginManager: Adding and executing plugin BTCPayServer.Plugins.FileSeller - 1.0.3 info: BTCPayServer.Plugins.PluginManager: Adding and executing plugin BTCPayServer.Plugins.NIP05 - 1.1.2 info: BTCPayServer.Plugins.PluginManager: Adding and executing plugin BTCPayServer.Plugins.DataErasure - 1.0.1 info: Configuration: Supported chains: BTC info: Configuration: BTC: Explorer url is http://nbxplorer:32838/ info: Configuration: BTC: Cookie file is /root/.nbxplorer/Main/.cookie info: Configuration: Network: Mainnet info: Configuration: LogFile: /datadir/btcpay.log info: Configuration: Log Level: Debug fail: BTCPayServer.Hosting.MigrationStartupTask: Error on the MigrationStartupTask System.Net.Sockets.SocketException (00000005, 0xFFFDFFFF): Name or service not known at System.Net.Dns.GetHostEntryOrAddressesCore(String hostName, Boolean justAddresses, AddressFamily addressFamily, Nullable1 startingTimestamp) at System.Net.Dns.<>c.b33_0(Object s, Int64 startingTimestamp) at System.Net.Dns.<>c__DisplayClass39_0`1.b0(Task , Object ) at System.Threading.Tasks.ContinuationResultTaskFromTask1.InnerInvoke() at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location --- at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread) --- End of stack trace from previous location --- at Npgsql.TaskTimeoutAndCancellation.ExecuteAsync(Func2 getTaskFunc, NpgsqlTimeout timeout, CancellationToken cancellationToken) at Npgsql.TaskTimeoutAndCancellation.ExecuteAsync[TResult](Func2 getTaskFunc, NpgsqlTimeout timeout, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.ConnectAsync(NpgsqlTimeout timeout, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.RawOpen(SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken, Boolean isFirstAttempt) at Npgsql.Internal.NpgsqlConnector.<Open>g__OpenCore|213_1(NpgsqlConnector conn, SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken, Boolean isFirstAttempt) at Npgsql.Internal.NpgsqlConnector.Open(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.UnpooledDataSource.Get(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.NpgsqlConnection.<Open>g__OpenAsync|42_0(Boolean async, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(Boolean errorsExpected, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(Boolean errorsExpected, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenAsync(CancellationToken cancellationToken, Boolean errorsExpected) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.Exists(Boolean async, CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.Exists(Boolean async, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Migrations.HistoryRepository.ExistsAsync(CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Migrations.HistoryRepository.GetAppliedMigrationsAsync(CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Migrations.Internal.NpgsqlMigrator.MigrateAsync(String targetMigration, CancellationToken cancellationToken) at BTCPayServer.Hosting.MigrationStartupTask.Migrate(CancellationToken cancellationToken) in /source/BTCPayServer/Hosting/MigrationStartupTask.cs:line 862 at BTCPayServer.Hosting.MigrationStartupTask.ExecuteAsync(CancellationToken cancellationToken) in /source/BTCPayServer/Hosting/MigrationStartupTask.cs:line 85 Unhandled exception. System.Net.Sockets.SocketException (00000005, 0xFFFDFFFF): Name or service not known at System.Net.Dns.GetHostEntryOrAddressesCore(String hostName, Boolean justAddresses, AddressFamily addressFamily, Nullable1 startingTimestamp) at System.Net.Dns.<>c.b33_0(Object s, Int64 startingTimestamp) at System.Net.Dns.<>c__DisplayClass39_0`1.b0(Task , Object ) at System.Threading.Tasks.ContinuationResultTaskFromTask1.InnerInvoke() at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location --- at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread) --- End of stack trace from previous location --- at Npgsql.TaskTimeoutAndCancellation.ExecuteAsync(Func2 getTaskFunc, NpgsqlTimeout timeout, CancellationToken cancellationToken) at Npgsql.TaskTimeoutAndCancellation.ExecuteAsync[TResult](Func`2 getTaskFunc, NpgsqlTimeout timeout, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.ConnectAsync(NpgsqlTimeout timeout, CancellationToken cancellationToken) at Npgsql.Internal.NpgsqlConnector.RawOpen(SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken, Boolean isFirstAttempt) at Npgsql.Internal.NpgsqlConnector.g__OpenCore|213_1(NpgsqlConnector conn, SslMode sslMode, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken, Boolean isFirstAttempt) at Npgsql.Internal.NpgsqlConnector.Open(NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.UnpooledDataSource.Get(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken) at Npgsql.NpgsqlConnection.g__OpenAsync|42_0(Boolean async, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(Boolean errorsExpected, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternalAsync(Boolean errorsExpected, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenAsync(CancellationToken cancellationToken, Boolean errorsExpected) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.Exists(Boolean async, CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlDatabaseCreator.Exists(Boolean async, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Migrations.HistoryRepository.ExistsAsync(CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.Migrations.HistoryRepository.GetAppliedMigrationsAsync(CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Migrations.Internal.NpgsqlMigrator.MigrateAsync(String targetMigration, CancellationToken cancellationToken) at BTCPayServer.Hosting.MigrationStartupTask.Migrate(CancellationToken cancellationToken) in /source/BTCPayServer/Hosting/MigrationStartupTask.cs:line 862 at BTCPayServer.Hosting.MigrationStartupTask.ExecuteAsync(CancellationToken cancellationToken) in /source/BTCPayServer/Hosting/MigrationStartupTask.cs:line 85 at Microsoft.AspNetCore.Hosting.WebHostExtensions.StartWithTasksAsync(IWebHost webHost, CancellationToken cancellationToken) in /source/BTCPayServer/Extensions/WebHostExtensions.cs:line 19 at BTCPayServer.Program.Main(String[] args) in /source/BTCPayServer/Program.cs:line 76 at BTCPayServer.Program.Main(String[] args) in /source/BTCPayServer/Program.cs:line 103 at BTCPayServer.Program.

(String[] args) bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) info: Configuration: Data Directory: /datadir info: Configuration: Configuration File: /datadir/Main/settings.config bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

NGINX docker logs --tail 100 nginx 2023/12/26 09:00:51 [notice] 55#55: gracefully shutting down 2023/12/26 09:00:51 [notice] 53#53: gracefully shutting down 2023/12/26 09:00:51 [notice] 54#54: exiting 2023/12/26 09:00:51 [notice] 52#52: exiting 2023/12/26 09:00:51 [notice] 54#54: exit 2023/12/26 09:00:51 [notice] 52#52: exit 2023/12/26 09:00:51 [notice] 55#55: exiting 2023/12/26 09:00:51 [notice] 53#53: exiting 2023/12/26 09:00:51 [notice] 55#55: exit 2023/12/26 09:00:51 [notice] 56#56: gracefully shutting down 2023/12/26 09:00:51 [notice] 57#57: gracefully shutting down 2023/12/26 09:00:51 [notice] 57#57: exiting 2023/12/26 09:00:51 [notice] 57#57: exit 2023/12/26 09:00:51 [notice] 53#53: exit 2023/12/26 09:00:51 [notice] 1#1: signal 17 (SIGCHLD) received from 52 2023/12/26 09:00:51 [notice] 1#1: worker process 52 exited with code 0 2023/12/26 09:00:51 [notice] 1#1: worker process 54 exited with code 0 2023/12/26 09:00:51 [notice] 1#1: worker process 55 exited with code 0 2023/12/26 09:00:51 [notice] 1#1: worker process 57 exited with code 0 2023/12/26 09:00:51 [notice] 1#1: signal 29 (SIGIO) received 2023/12/26 09:00:51 [notice] 1#1: signal 17 (SIGCHLD) received from 57 2023/12/26 09:00:51 [notice] 1#1: signal 17 (SIGCHLD) received from 53 2023/12/26 09:00:51 [notice] 1#1: worker process 53 exited with code 0 2023/12/26 09:00:51 [notice] 1#1: signal 29 (SIGIO) received 2023/12/26 09:00:53 [error] 56#56: 313 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://172.18.0.9:49392/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:00:53 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:00:53 [notice] 56#56: exiting 2023/12/26 09:00:53 [notice] 56#56: exit 2023/12/26 09:00:53 [notice] 1#1: signal 17 (SIGCHLD) received from 56 2023/12/26 09:00:53 [notice] 1#1: worker process 56 exited with code 0 2023/12/26 09:00:53 [notice] 1#1: signal 29 (SIGIO) received 2023/12/26 09:00:54 [error] 58#58: 315 no live upstreams while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://btcpay/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:00:54 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:00:57 [error] 59#59: 316 no live upstreams while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://btcpay/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:00:57 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:01:05 [error] 60#60: 317 no live upstreams while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://btcpay/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:01:05 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:01:08 [error] 61#61: 318 no live upstreams while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://btcpay/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:01:08 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:01:09 [error] 62#62: 319 no live upstreams while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://btcpay/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:01:09 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:01:11 [error] 63#63: 320 no live upstreams while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://btcpay/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:01:11 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:01:12 [error] 58#58: 321 no live upstreams while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://btcpay/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:01:12 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:01:16 [error] 58#58: 322 no live upstreams while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://btcpay/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:01:16 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:01:18 [error] 58#58: 323 no live upstreams while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://btcpay/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:01:18 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:01:19 [notice] 1#1: signal 1 (SIGHUP) received, reconfiguring 2023/12/26 09:01:19 [notice] 1#1: reconfiguring 2023/12/26 09:01:19 [notice] 1#1: using the "epoll" event method 2023/12/26 09:01:19 [notice] 1#1: start worker processes 2023/12/26 09:01:19 [notice] 1#1: start worker process 64 2023/12/26 09:01:19 [notice] 1#1: start worker process 65 2023/12/26 09:01:19 [notice] 1#1: start worker process 66 2023/12/26 09:01:19 [notice] 1#1: start worker process 67 2023/12/26 09:01:19 [notice] 1#1: start worker process 68 2023/12/26 09:01:19 [notice] 1#1: start worker process 69 2023/12/26 09:01:19 [notice] 58#58: gracefully shutting down 2023/12/26 09:01:19 [notice] 59#59: gracefully shutting down 2023/12/26 09:01:19 [notice] 59#59: exiting 2023/12/26 09:01:19 [notice] 60#60: gracefully shutting down 2023/12/26 09:01:19 [notice] 62#62: gracefully shutting down 2023/12/26 09:01:19 [notice] 60#60: exiting 2023/12/26 09:01:19 [notice] 62#62: exiting 2023/12/26 09:01:19 [notice] 58#58: exiting 2023/12/26 09:01:19 [notice] 60#60: exit 2023/12/26 09:01:19 [notice] 59#59: exit 2023/12/26 09:01:19 [notice] 58#58: exit 2023/12/26 09:01:19 [notice] 62#62: exit 2023/12/26 09:01:19 [notice] 63#63: gracefully shutting down 2023/12/26 09:01:19 [notice] 63#63: exiting 2023/12/26 09:01:19 [notice] 63#63: exit 2023/12/26 09:01:19 [notice] 61#61: gracefully shutting down 2023/12/26 09:01:19 [notice] 61#61: exiting 2023/12/26 09:01:19 [notice] 61#61: exit 2023/12/26 09:01:19 [notice] 1#1: signal 17 (SIGCHLD) received from 59 2023/12/26 09:01:19 [notice] 1#1: worker process 58 exited with code 0 2023/12/26 09:01:19 [notice] 1#1: worker process 59 exited with code 0 2023/12/26 09:01:19 [notice] 1#1: worker process 62 exited with code 0 2023/12/26 09:01:19 [notice] 1#1: signal 29 (SIGIO) received 2023/12/26 09:01:19 [notice] 1#1: signal 17 (SIGCHLD) received from 62 2023/12/26 09:01:19 [notice] 1#1: signal 17 (SIGCHLD) received from 60 2023/12/26 09:01:19 [notice] 1#1: worker process 60 exited with code 0 2023/12/26 09:01:19 [notice] 1#1: signal 29 (SIGIO) received 2023/12/26 09:01:19 [notice] 1#1: signal 17 (SIGCHLD) received from 63 2023/12/26 09:01:19 [notice] 1#1: worker process 63 exited with code 0 2023/12/26 09:01:19 [notice] 1#1: signal 29 (SIGIO) received 2023/12/26 09:01:19 [notice] 1#1: signal 17 (SIGCHLD) received from 61 2023/12/26 09:01:19 [notice] 1#1: worker process 61 exited with code 0 2023/12/26 09:01:19 [notice] 1#1: signal 29 (SIGIO) received 2023/12/26 09:01:19 [error] 64#64: 324 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://172.18.0.9:49392/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:01:19 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" localhost 172.18.0.1 - - [26/Dec/2023:09:01:20 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:01:20 [error] 65#65: 326 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://172.18.0.9:49392/", host: "localhost" 2023/12/26 09:01:24 [error] 66#66: 328 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://172.18.0.9:49392/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:01:24 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0" 2023/12/26 09:01:26 [error] 67#67: 330 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.1, server: pay.fomozon.com, request: "GET / HTTP/1.1", upstream: "http://172.18.0.9:49392/", host: "localhost" localhost 172.18.0.1 - - [26/Dec/2023:09:01:26 +0000] "GET / HTTP/1.1" 502 150 "-" "curl/7.68.0"

I also tried to run the backup script to migrate to a fresh server, but the database in the backup file is empty.

I can access the RockPro via SSH and all data seems to still be there.

Any help is greatly appreciated.

NicolasDorier commented 6 months ago

Can you check the postgres logs? looking at it, it seems postgres isn't starting.

gobrrrme commented 6 months ago

Thanks for the quick response.

Here's the output:

docker logs --tail 100 generated_postgres_1

PostgreSQL Database directory appears to contain a database; Skipping initialization

PostgreSQL Database directory appears to contain a database; Skipping initialization

PostgreSQL Database directory appears to contain a database; Skipping initialization

PostgreSQL Database directory appears to contain a database; Skipping initialization

PostgreSQL Database directory appears to contain a database; Skipping initialization

PostgreSQL Database directory appears to contain a database; Skipping initialization

PostgreSQL Database directory appears to contain a database; Skipping initialization

PostgreSQL Database directory appears to contain a database; Skipping initialization

gobrrrme commented 6 months ago

Fixed! Thanks for your help. Massively appreciated, especially at this time of the year!

NicolasDorier commented 6 months ago

so the problem was postgres image corrupted.

btcpay-down.sh
docker container prune
docker rmi btcpayserver/postgres:13.13
docker pull btcpayserver/postgres:13.13
btcapy-up.sh

Seems to have fixed the issue. Unclear what happened.