oracle / oracle-r2dbc

R2DBC Driver for Oracle Database
https://oracle.com
Other
193 stars 40 forks source link

Issue with the OracleR2dbcOptions.Executor option #108

Closed htejwani closed 1 year ago

htejwani commented 1 year ago

We are facing some issues with the Executor option set to default in our containerized app which has 2 cpu cores. All the DB calls in our app hang after approx. 10 mins of load test. There is no log to indicate what's gone wrong. We do not have the means yet to pull a thread dump. The cpu and memory usage is normal and the connections in the pool are well within the limit. This issue occurs when: a) The OracleR2dbcIptions.EXECUTOR is not set, left to be default OR b) Set to Executors.newWorkStealingPool(2)

When we set this option to "new ForkJoinPool(4)", our load tests seem to run fine. The one thing that I noticed with this option is that the asyncMode for the ForkJoinPool is set to false.

Michael-A-McMahon commented 1 year ago

Thanks for reporting this.

This could be a deadlock induced by threads contending for Oracle JDBC connection lock. If an Oracle JDBC connection is executing an async call, and a thread calls a method on that connection, that thread is blocked until the async call completes. This leads to deadlocks where all threads are blocked on method calls, and no threads are left to complete the async call.

Since the 0.4.0 release, Oracle R2DBC is designed to avoid this kind of deadlock. It should be the case that any JDBC method call gets pushed to AsyncLock.

It's possible that I've missed a JDBC call somewhere, and have not pushed it to the AsyncLock. This is my best guess as to the root cause of this issue.

A thread dump would be the best way to figure out where the issue is. Is it possible to provide that?

Michael-A-McMahon commented 1 year ago

I'm closing this issue now. Although I'm still a bit paranoid about having missed a blocking call somewhere.