search

brettwooldridge / HikariCP

光 HikariCP・A solid, high-performance, JDBC connection pool at last.
Apache License 2.0
19.96k stars 2.93k forks source link

java.lang.OutOfMemoryError: unable to create new native thread #1948

Closed redeyed-13 closed 2 years ago

redeyed-13 commented 2 years ago

after replacing an old version of hibernate and C3P0 with Hibernate 5.6.4-Final and HikariCP 4.0.3 this exception is killing us:

2022-07-05 06:27:34.136 ****>> Exception in thread "*********************" java.lang.OutOfMemoryError: unable to create new native thread
2022-07-05 06:27:34.247 ****>>  at java.lang.Thread.start0(Native Method)
2022-07-05 06:27:34.247 ****>>  at java.lang.Thread.start(Thread.java:719)
2022-07-05 06:27:34.247 ****>>  at oracle.jdbc.driver.PhysicalConnection.pingDatabase(PhysicalConnection.java:6162)
2022-07-05 06:27:34.247 ****>>  at oracle.jdbc.driver.PhysicalConnection.isValid(PhysicalConnection.java:10887)
2022-07-05 06:27:34.247 ****>>  at oracle.jdbc.driver.PhysicalConnection.isValid(PhysicalConnection.java:10859)
2022-07-05 06:27:34.247 ****>>  at com.zaxxer.hikari.pool.PoolBase.isConnectionAlive(PoolBase.java:161)
2022-07-05 06:27:34.247 ****>>  at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:186)
2022-07-05 06:27:34.247 ****>>  at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:162)
2022-07-05 06:27:34.247 ****>>  at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:100)
2022-07-05 06:27:34.247 ****>>  at org.hibernate.hikaricp.internal.HikariCPConnectionProvider.getConnection(HikariCPConnectionProvider.java:77)
2022-07-05 06:27:34.247 ****>>  at org.hibernate.internal.NonContextualJdbcConnectionAccess.obtainConnection(NonContextualJdbcConnectionAccess.java:38)
2022-07-05 06:27:34.247 ****>>  at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:108)
2022-07-05 06:27:34.247 ****>>  at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:138)
2022-07-05 06:27:34.247 ****>>  at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getConnectionForTransactionManagement(LogicalConnectionManagedImpl.java:276)
2022-07-05 06:27:34.247 ****>>  at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.begin(LogicalConnectionManagedImpl.java:284)
2022-07-05 06:27:34.247 ****>>  at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.begin(JdbcResourceLocalTransactionCoordinatorImpl.java:246)
2022-07-05 06:27:34.247 ****>>  at org.hibernate.engine.transaction.internal.TransactionImpl.begin(TransactionImpl.java:83)
2022-07-05 06:27:34.247 ****>>  at org.hibernate.internal.AbstractSharedSessionContract.beginTransaction(AbstractSharedSessionContract.java:503)
2022-07-05 06:27:34.247 ****>>  at some.where.in.the.code.beginTransaction(***************:123)

in addition to the connection parameters, we have specified only these parameters: hibernate.hikari.autoCommit=false hibernate.hikari.maximumPoolSize=10 hibernate.hikari.idleTimeout=600000 hibernate.hikari.leakDetectionThreshold=15000

Any idea? Thanks in advance.

lfbayer commented 2 years ago

You should attach a profiler or take a heap dump and identify what is consuming the memory thus causing the OutOfMemoryException. It is very unlikely to be caused by HikariCP.

You can have java automatically create a heap dump file for you by adding this option to the java command line: -XX:+HeapDumpOnOutOfMemoryError

If after doing this you can identify HikariCP as being the cause, please reopen this ticket with the details of what objects were leaked and causing the memory issue.

frolovgo commented 1 year ago

The main problem is that "java.lang.OutOfMemoryError: unable to create new native thread" is temporary (it corresponds to similar messages in the log [warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached ), but after this the connection in the pool remains "Active" forever. And within a short period of time, all available connections in the pool may be exhausted.

Is it possible that the exception "java.lang.OutOfMemoryError: unable to create new native thread" in 

...
com.zaxxer.hikari.pool.PoolBase.isConnectionAlive(PoolBase.java:16)
com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:186))
...

did not cause the connection to hang in Active-status? Thanks in advance.

Additional Information: hikariCP: 5.0.1 openjdk: 11.0.20 ojdbc8: 19.8.0

P.S. Fix: https://bugs.openjdk.org/browse/JDK-8268773 did not solve the problem completely.