Provide method(s) to evict connections currently in pool

[ttddyy]

Method to evict(invalidate/purge) connections currently in pool. In-use connections need to be invalidated after it is released. Or user should be able to define the behavior by passing parameter or provide separate methods for different behavior.

[nishant-1984]

Is this being implemented? There are serious connection leak issues with library

[mp911de]

There are no proper methods to signal a full eviction to the pool. We should file a ticket in https://github.com/reactor/reactor-pool/issues.

[mbanaszkiewicz-vonage]

Probably related to this issue.

Sometimes I'm seeing the following warning log from netty, after enabling debug it looks like its preceded by connection release: {"timestamp":"2020-07-01T13:14:36.654+0000","level":"DEBUG","thread":"reactor-tcp-epoll-1","message":"Releasing R2DBC Connection [PooledConnection[dev.miku.r2dbc.mysql.MySqlConnection@1c185c05]] after transaction","correlationId":"c46aa7b3-79b3-5080-ad3c-ee494705c4b8"} **{"timestamp":"2020-07-01T13:14:45.064+0000","level":"WARN","thread":"reactor-tcp-epoll-1","message":"Connection has been closed by peer","correlationId":"c46aa7b3-79b3-5080-ad3c-ee494705c4b8"}**

Our r2dbc-pool settings are as follows:

          max-size: 20
          initial-size: 1
          max-life-time: 280 seconds
          max-idle-time: 240 seconds
          max-acquire-time: 30 seconds

Our Mysql database has 300 seconds max connection lifetime. We checked connection list on mysql side, and turns out r2dbc-pool connections are sometimes not closed properly on pool side after 280s, and mysql server has to clean them up after 300s. We are using spring boot 2.3.0, spring-boot-starter-data-r2dbc 2.3.0, r2dbc-pool 0.8.2, reactor-pool 0.1.3, r2dbc-mysql 0.8.1 - also tried bumping up r2dbc-pool & reactor-pool to newest - didn't change anything.

Is there any ongoing work which might improve the situation?

[mbanaszkiewicz-vonage]

I've prepared a simple spring boot app which reproduces the problem and makes "connection closed by peer" logs visible: https://github.com/mbanaszkiewicz-vonage/r2dbc-testing

README.md contains very short instruction what to do.

[mp911de]

We're using Reactor Pool to provide general object pooling functionality. Eviction of connections only happens if there's activity on the pool. There's no background thread that evicts connections actively. Instead, the application must either borrow or return connections. Can you recheck whether this works for you if your application has pool activity if you exceed 280 seconds?

[mbanaszkiewicz-vonage]

All right, thanks for clarification. I've verified this by generating constant load, indeed pool activity causes the connections to be evicted properly. @mp911de there are no plans to change this behavior and introducing something automatic?

[mp911de]

It would make sense to ask reactor-pool to provide background eviction so that TTLs can be considered properly. I think we cannot do anything about automated eviction from a R2DBC pool perspective.

[petromir]

Isn't this covered with https://github.com/r2dbc/r2dbc-pool/issues/115?

[mp911de]

Actually, not, because reactor.pool.Pool doesn't expose an evictAll method.

[ojh3636]

@petromir I think that's because, r2dbc pool set backgroundEvictionInterval of reactor pool as same as maxIdleTime when it created. So, when creating reactor pool, it start's eviction and then do next eviction after maxIdleTime (in your case 240s). But your first connections are acquired after the reactor pool initialized, your first connections has no chance to properly evicted by reactor pool.

0s -> 1s -> 240s -> 300s ->
(pool created, eviction) -> first connection acquired -> first eviction run (but no evict), -> connection closed by db

So I recommend to set (backgroundEvictionInterval + maxIdleTime) < wait_time of db, then you can solve your problem

[restrepo86Me]

Hi, I am currently having this problem, could you put an example of configuration based on what you just mentioned (backgroundEvictionInterval + maxIdleTime) considering that wait_time of db is 20 minutes ?