brettwooldridge / HikariCP

光 HikariCP・A solid, high-performance, JDBC connection pool at last.
Apache License 2.0
19.87k stars 2.92k forks source link

Randomly HikariPool pool becomes zero and HikariCP is not renewing/creating new connections #1799

Closed knop-ster closed 3 years ago

knop-ster commented 3 years ago

I have had the same problem with my Spring Boot application as in ticket #1256 for a long time. Only now did I come across this ticket.

Hikari closes connections because maxLifeTime has been reached. When things go really bad, Hikari is unable to establish new connections. Allegedly because the login with the user fails.

At the end of the log, however, you can clearly see that the connection can then be established. So the password seems to be correct. Do you have any idea what the problem could be?

Here are some environment details:

Spring boot: 2.5.1
JDK version: 15.0.2
Database: postgresql 13.3
Docker: 20.10.7
Docker-Compose: 1.21.0

Thats my current hikari configuration:

HikariPool-1 - configuration:
allowPoolSuspension................................false
autoCommit................................false
catalog................................none
connectionInitSql................................none
connectionTestQuery................................none
connectionTimeout................................30000
dataSource................................none
dataSourceClassName................................none
dataSourceJNDI................................none
dataSourceProperties................................{password=<masked>}
driverClassName................................"org.postgresql.Driver"
exceptionOverrideClassName................................none
healthCheckProperties................................{}
healthCheckRegistry................................none
idleTimeout................................600000
initializationFailTimeout................................1
isolateInternalQueries................................false
jdbcUrl................................jdbc:postgresql://database:5432/postgres
keepaliveTime................................0
leakDetectionThreshold................................0
maxLifetime................................1800000
maximumPoolSize................................50
metricRegistry................................none
metricsTrackerFactory................................none
minimumIdle................................10
password................................<masked>
poolName................................"HikariPool-1"
readOnly................................false
registerMbeans................................false
scheduledExecutor................................none
schema................................none
threadFactory................................internal
transactionIsolation................................default
username................................"postgres"
validationTimeout................................5000

Here is the stack trace

2021-07-02 17:57:21,593 [HikariPool-1 connection closer] DEBUG com.zaxxer.hikari.pool.PoolBase: HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@378a2e49: (connection has passed maxLifetime)
2021-07-02 17:57:21,598 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Cannot acquire connection from data source
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
        at org.postgresql.Driver.makeConnection(Driver.java:465)
        at org.postgresql.Driver.connect(Driver.java:264)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
        at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
        at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
        at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:726)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:831)
2021-07-02 17:57:21,852 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Cannot acquire connection from data source
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
        at org.postgresql.Driver.makeConnection(Driver.java:465)
        at org.postgresql.Driver.connect(Driver.java:264)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
        at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
        at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
        at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:726)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:831)
2021-07-02 17:57:22,231 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Cannot acquire connection from data source
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
        at org.postgresql.Driver.makeConnection(Driver.java:465)
        at org.postgresql.Driver.connect(Driver.java:264)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
        at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
        at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
        at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:726)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:831)
2021-07-02 17:57:22,797 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Cannot acquire connection from data source
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
        at org.postgresql.Driver.makeConnection(Driver.java:465)
        at org.postgresql.Driver.connect(Driver.java:264)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
        at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
        at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
        at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:726)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:831)
2021-07-02 17:57:23,644 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Cannot acquire connection from data source
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
        at org.postgresql.Driver.makeConnection(Driver.java:465)
        at org.postgresql.Driver.connect(Driver.java:264)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
        at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
        at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
        at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:726)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:831)
2021-07-02 17:57:24,236 [HikariPool-1 connection closer] DEBUG com.zaxxer.hikari.pool.PoolBase: HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@478b92fe: (connection has passed maxLifetime)
2021-07-02 17:57:24,236 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 0, queue 1
2021-07-02 17:57:24,913 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Cannot acquire connection from data source
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
        at org.postgresql.Driver.makeConnection(Driver.java:465)
        at org.postgresql.Driver.connect(Driver.java:264)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
        at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
        at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
        at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:726)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:831)
2021-07-02 17:57:25,202 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 0, queue 2
2021-07-02 17:57:25,202 [HikariPool-1 connection closer] DEBUG com.zaxxer.hikari.pool.PoolBase: HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@33b5ac60: (connection has passed maxLifetime)
2021-07-02 17:57:25,307 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 0, queue 3
2021-07-02 17:57:25,307 [HikariPool-1 connection closer] DEBUG com.zaxxer.hikari.pool.PoolBase: HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@211518f3: (connection has passed maxLifetime)
2021-07-02 17:57:26,813 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Cannot acquire connection from data source
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
        at org.postgresql.Driver.makeConnection(Driver.java:465)
        at org.postgresql.Driver.connect(Driver.java:264)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
        at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
        at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
        at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:726)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:831)
2021-07-02 17:57:27,110 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 0, queue 4
2021-07-02 17:57:27,110 [HikariPool-1 connection closer] DEBUG com.zaxxer.hikari.pool.PoolBase: HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@1698db3d: (connection has passed maxLifetime)
2021-07-02 17:57:29,661 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Cannot acquire connection from data source
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
        at org.postgresql.Driver.makeConnection(Driver.java:465)
        at org.postgresql.Driver.connect(Driver.java:264)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
        at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
        at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
        at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:726)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:831)
2021-07-02 17:57:31,603 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 0, queue 5
2021-07-02 17:57:31,603 [HikariPool-1 connection closer] DEBUG com.zaxxer.hikari.pool.PoolBase: HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@76b72472: (connection has passed maxLifetime)
2021-07-02 17:57:31,651 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 0, queue 6
2021-07-02 17:57:31,651 [HikariPool-1 connection closer] DEBUG com.zaxxer.hikari.pool.PoolBase: HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@59c3f0c3: (connection has passed maxLifetime)
2021-07-02 17:57:32,062 [HikariPool-1 connection closer] DEBUG com.zaxxer.hikari.pool.PoolBase: HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@394dc710: (connection has passed maxLifetime)
2021-07-02 17:57:32,062 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 0, queue 7
2021-07-02 17:57:33,933 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Cannot acquire connection from data source
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
        at org.postgresql.Driver.makeConnection(Driver.java:465)
        at org.postgresql.Driver.connect(Driver.java:264)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
        at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
        at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
        at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:726)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:831)
2021-07-02 17:57:34,713 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Before cleanup stats (total=2, active=0, idle=2, waiting=0)
2021-07-02 17:57:34,714 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - After cleanup  stats (total=2, active=0, idle=2, waiting=0)
2021-07-02 17:57:34,714 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Fill pool skipped, pool is at sufficient level.
2021-07-02 17:57:40,337 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Cannot acquire connection from data source
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
        at org.postgresql.Driver.makeConnection(Driver.java:465)
        at org.postgresql.Driver.connect(Driver.java:264)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
        at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
        at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
        at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:726)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:831)
2021-07-02 17:57:49,939 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Cannot acquire connection from data source
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
        at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:613)
        at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:161)
        at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
        at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
        at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:223)
        at org.postgresql.Driver.makeConnection(Driver.java:465)
        at org.postgresql.Driver.connect(Driver.java:264)
        at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
        at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364)
        at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
        at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476)
        at com.zaxxer.hikari.pool.HikariPool.access$100(HikariPool.java:71)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:726)
        at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:712)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base/java.lang.Thread.run(Thread.java:831)
2021-07-02 17:57:59,944 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@5446ee3a
2021-07-02 17:57:59,949 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@5a811ac2
2021-07-02 17:57:59,950 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - After adding stats (total=4, active=0, idle=4, waiting=0)
2021-07-02 17:57:59,955 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@5ebe854a
2021-07-02 17:57:59,955 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - After adding stats (total=5, active=0, idle=5, waiting=0)
2021-07-02 17:57:59,961 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@1bbc10f9
2021-07-02 17:57:59,961 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - After adding stats (total=6, active=0, idle=6, waiting=0)
2021-07-02 17:57:59,969 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@78e80a8a
2021-07-02 17:57:59,970 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - After adding stats (total=7, active=0, idle=7, waiting=0)
2021-07-02 17:57:59,975 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@2a6d485f
2021-07-02 17:57:59,976 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - After adding stats (total=8, active=0, idle=8, waiting=0)
2021-07-02 17:57:59,983 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@d30c8d
2021-07-02 17:57:59,983 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - After adding stats (total=9, active=0, idle=9, waiting=0)
2021-07-02 17:57:59,992 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@7b93efb6
2021-07-02 17:57:59,992 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - After adding stats (total=10, active=0, idle=10, waiting=0)
2021-07-02 17:58:01,028 [app-task-pool-87] DEBUG c.zaxxer.hikari.pool.ProxyConnection: HikariPool-1 - Executed rollback on connection org.postgresql.jdbc.PgConnection@6310d8ba due to dirty commit state on close().
2021-07-02 17:58:01,031 [app-task-pool-22] DEBUG c.zaxxer.hikari.pool.ProxyConnection: HikariPool-1 - Executed rollback on connection org.postgresql.jdbc.PgConnection@5ebe854a due to dirty commit state on close().
2021-07-02 17:58:01,034 [app-task-pool-85] DEBUG c.zaxxer.hikari.pool.ProxyConnection: HikariPool-1 - Executed rollback on connection org.postgresql.jdbc.PgConnection@a7a7029 due to dirty commit state on close().
2021-07-02 17:58:01,035 [app-task-pool-86] DEBUG c.zaxxer.hikari.pool.ProxyConnection: HikariPool-1 - Executed rollback on connection org.postgresql.jdbc.PgConnection@5446ee3a due to dirty commit state on close()
knop-ster commented 3 years ago

Here some supplementary information.

My current docker-compose.yml

version: '2'

networks:
  intern:
    external: true
  web:
    external: true

services:
  database:
    image: postgres:13.3
    restart: always
    ports:
      - 5432
    environment:
      - POSTGRES_PASSWORD=4nu8FuFcrnrjWqtzpCyFgvuCrPCzRHuTmr4MxHbwFyudB7Be6g8VgfaUDQQGc398
    networks:
      - intern
    volumes:
      - /var/docker/postgresql/data:/var/lib/postgresql/data

  app:
    image: myImage
    restart: always
    entrypoint:
      - 'java'
      - '-jar'
      - 'app.jar'
    ports:
      - 8443
    depends_on:
      - database
    networks:
      - intern
      - web
    environment:
      - SPRING_PROFILES_ACTIVE=stage
      - SPRING_LIQUIBASE_ENABLED=false
      - SPRING_DATASOURCE_URL=jdbc:postgresql://database:5432/postgres
      - SPRING_DATASOURCE_USERNAME=postgres
      - SPRING_DATASOURCE_PASSWORD=4nu8FuFcrnrjWqtzpCyFgvuCrPCzRHuTmr4MxHbwFyudB7Be6g8VgfaUDQQGc398
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro

configuration of my docker networks

[
    {
        "Name": "intern",
        "Id": "76f8a09726789c351db97aa1fea68f1d02d0a1390a6ac384b63a666f212e8b37",
        "Created": "2021-07-04T16:34:42.641320042+02:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.21.0.0/16",
                    "Gateway": "172.21.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "2d9e0306c9b8e39c46169d3fecfade08ed7d12fa0510b07113deea22369bd48c": {
                "Name": "stage_database_1",
                "EndpointID": "f9c6a01a1f7a80eb727d84a0baca0fbe5d66c8c799cfa61982d87629181b7633",
                "MacAddress": "02:42:ac:15:00:04",
                "IPv4Address": "172.21.0.4/16",
                "IPv6Address": ""
            },
            "b89adb628a8042cd933e6ef4da0d304d81fd86b46c05321d8846c4a686fb254b": {
                "Name": "stage_app_1",
                "EndpointID": "59aca6671c06dbf6f81ca4a13b58abd23a8a7496b885f57201e37637030d7ab8",
                "MacAddress": "02:42:ac:15:00:05",
                "IPv4Address": "172.21.0.5/16",
                "IPv6Address": ""
            }
        },
        "Options": {
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.driver.mtu": "9216"
        },
        "Labels": {}
    }
]
[
    {
        "Name": "web",
        "Id": "94eb39f3770cfbe87554def0ce8db7bb6925ff528fd00295b78b8c42c6fa26ce",
        "Created": "2020-12-22T10:28:20.40424673+01:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.19.0.0/16",
                    "Gateway": "172.19.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "b89adb628a8042cd933e6ef4da0d304d81fd86b46c05321d8846c4a686fb254b": {
                "Name": "stage_app_1",
                "EndpointID": "718d38e49509e4f9a11fce853f2c574a1b41cbe49df45440d0fb226ec14c277e",
                "MacAddress": "02:42:ac:13:00:04",
                "IPv4Address": "172.19.0.4/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]
2021-07-04 17:11:31,004 [community-event-manager-task-pool-83] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Timeout failure stats (total=0, active=0, idle=0, waiting=5)
2021-07-04 17:11:31,005 [community-event-manager-task-pool-83] WARN  o.h.e.jdbc.spi.SqlExceptionHelper: SQL Error: 0, SQLState: 28P01
2021-07-04 17:11:31,006 [community-event-manager-task-pool-83] ERROR o.h.e.jdbc.spi.SqlExceptionHelper: HikariPool-1 - Connection is not available, request timed out after 30003ms.
2021-07-04 17:11:31,006 [community-event-manager-task-pool-83] ERROR o.h.e.jdbc.spi.SqlExceptionHelper: FATAL: password authentication failed for user "postgres"
2021-07-04 17:11:31,007 [community-event-manager-task-pool-83] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 6, queue 10
2021-07-04 17:11:31,010 [community-event-manager-task-pool-84] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Timeout failure stats (total=0, active=0, idle=0, waiting=5)
2021-07-04 17:11:31,011 [community-event-manager-task-pool-84] WARN  o.h.e.jdbc.spi.SqlExceptionHelper: SQL Error: 0, SQLState: 28P01
2021-07-04 17:11:31,010 [community-event-manager-task-pool-21] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Timeout failure stats (total=0, active=0, idle=0, waiting=4)
2021-07-04 17:11:31,010 [community-event-manager-task-pool-42] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Timeout failure stats (total=0, active=0, idle=0, waiting=3)
2021-07-04 17:11:31,011 [community-event-manager-task-pool-21] WARN  o.h.e.jdbc.spi.SqlExceptionHelper: SQL Error: 0, SQLState: 28P01
2021-07-04 17:11:31,012 [community-event-manager-task-pool-42] WARN  o.h.e.jdbc.spi.SqlExceptionHelper: SQL Error: 0, SQLState: 28P01
2021-07-04 17:11:31,011 [community-event-manager-task-pool-84] ERROR o.h.e.jdbc.spi.SqlExceptionHelper: HikariPool-1 - Connection is not available, request timed out after 30002ms.
2021-07-04 17:11:31,013 [community-event-manager-task-pool-84] ERROR o.h.e.jdbc.spi.SqlExceptionHelper: FATAL: password authentication failed for user "postgres"
2021-07-04 17:11:31,012 [community-event-manager-task-pool-21] ERROR o.h.e.jdbc.spi.SqlExceptionHelper: HikariPool-1 - Connection is not available, request timed out after 30002ms.
2021-07-04 17:11:31,013 [community-event-manager-task-pool-42] ERROR o.h.e.jdbc.spi.SqlExceptionHelper: HikariPool-1 - Connection is not available, request timed out after 30002ms.
2021-07-04 17:11:31,014 [community-event-manager-task-pool-21] ERROR o.h.e.jdbc.spi.SqlExceptionHelper: FATAL: password authentication failed for user "postgres"
2021-07-04 17:11:31,014 [community-event-manager-task-pool-42] ERROR o.h.e.jdbc.spi.SqlExceptionHelper: FATAL: password authentication failed for user "postgres"
2021-07-04 17:11:31,016 [community-event-manager-task-pool-21] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 4, queue 10
2021-07-04 17:11:31,016 [community-event-manager-task-pool-42] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 6, queue 10
2021-07-04 17:11:31,016 [community-event-manager-task-pool-84] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 5, queue 10
2021-07-04 17:11:40,001 [community-event-manager-task-pool-85] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Timeout failure stats (total=0, active=0, idle=0, waiting=5)
2021-07-04 17:11:40,002 [community-event-manager-task-pool-85] WARN  o.h.e.jdbc.spi.SqlExceptionHelper: SQL Error: 0, SQLState: 28P01
2021-07-04 17:11:40,002 [community-event-manager-task-pool-85] ERROR o.h.e.jdbc.spi.SqlExceptionHelper: HikariPool-1 - Connection is not available, request timed out after 30001ms.
2021-07-04 17:11:40,002 [community-event-manager-task-pool-85] ERROR o.h.e.jdbc.spi.SqlExceptionHelper: FATAL: password authentication failed for user "postgres"
2021-07-04 17:11:40,003 [community-event-manager-task-pool-85] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 6, queue 10

Until shortly after that happens

2021-07-04 17:11:50,447 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@2b4f8d8d
2021-07-04 17:11:50,447 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - After adding stats (total=2, active=1, idle=1, waiting=0)
2021-07-04 17:11:50,461 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@400bb413
2021-07-04 17:11:50,461 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - After adding stats (total=3, active=1, idle=2, waiting=0)
2021-07-04 17:11:50,470 [community-event-manager-task-pool-42] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 1, queue 7
2021-07-04 17:11:50,471 [community-event-manager-task-pool-85] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 2, queue 7
2021-07-04 17:11:50,472 [community-event-manager-task-pool-84] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Add connection elided, waiting 3, queue 7
2021-07-04 17:11:50,478 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@16564673
2021-07-04 17:11:50,479 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - After adding stats (total=4, active=4, idle=0, waiting=2)

I am out of ideas to analyze the problem.