r2dbc / r2dbc-pool

Connection Pooling for Reactive Relational Database Connectivity
https://r2dbc.io
Apache License 2.0
337 stars 55 forks source link

possible r2dbc-pool leak #102

Closed climber73 closed 3 years ago

climber73 commented 3 years ago

Bug Report

Versions

Current Behavior

Stack trace ``` // your stack trace here ```

Table schema

Input Code ```sql -- your SQL here; ```

Steps to reproduce

Input Code ```java // your code here; ```

Expected behavior/code

It looks like there is a subtle issue somewhere in r2dbc-pool, r2dbc-postgresql or even in projectreactor. Here is a small project to reproduce the issue. Please help with investigation of it. I understand that the test looks like a bit far-fetched, but it models our case we faced during development of billing application with CockroachDB as its database.

Possible Solution

Additional context

climber73 commented 3 years ago

Here is short description of the issue:

As I mentioned before there is a simple project which is a HTTP server with one method /payment. When the server gets the payment request, it tries to execute 3 SQL queries (select, update and insert) inside a transaction in serializable level of isolation. The update query is likely going to be failed under heavy load because of 40001 error. The server is designed to retry such errors (10 times max).

Also that project has a load test, performing concurrent HTTP requests of the server. The test makes 2 phases: (1) preliminary phase - to assess needed time and (2) main phase where the same load test is run but for the half of the time (to get HTTP client close their connections). After the second phase the server never gets all DB connections idle in r2dbc connection pool. Some of them gets acquired forever (metric r2dbc.pool.acquired never gets 0 and that connections can't be used any more).

Because of that issue we had to switch to JDBC driver and now there are tons of threads which are blocked most of the time.

Kindly ask to help with that problem.

mp911de commented 3 years ago

It would make sense to reproduce the issue in an isolated environment. Debugging the pool through complex application logic is impossible for us.

mp911de commented 3 years ago

That should be addressed with #115. Feel free to provide additional details to reopen this ticket if the problem persists.