search

spring-projects / spring-data-r2dbc

Provide support to increase developer productivity in Java when using Reactive Relational Database Connectivity. Uses familiar Spring concepts such as a DatabaseClient for core API usage and lightweight repository style data access.
Apache License 2.0
708 stars 133 forks source link

TimeoutException on connection acquisition after server failure #362

Closed itzikiusa closed 4 years ago

itzikiusa commented 4 years ago

Hi, using r2dbc connection pool and when a DB server fails for some reason, seems like it is not able to reconnect to it

this is the exception:


Caused by: org.springframework.dao.DataAccessResourceFailureException: Failed to obtain R2DBC Connection; nested exception is java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 30000ms in 'peek' (and no fallback has been configured)
at org.springframework.data.r2dbc.connectionfactory.ConnectionFactoryUtils.lambda$getConnection$0(ConnectionFactoryUtils.java:70) ~[spring-data-r2dbc-1.0.0.RELEASE.jar!/:1.0.0.RELEASE]
at reactor.core.publisher.Mono.lambda$onErrorMap$30(Mono.java:3270) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:88) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:100) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:185) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:251) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.FluxRetry$RetrySubscriber.onError(FluxRetry.java:88) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.SerializedSubscriber.onError(SerializedSubscriber.java:114) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.handleTimeout(FluxTimeout.java:289) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.doTimeout(FluxTimeout.java:274) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.FluxTimeout$TimeoutTimeoutSubscriber.onNext(FluxTimeout.java:396) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.StrictSubscriber.onNext(StrictSubscriber.java:89) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:73) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.publisher.MonoDelay$MonoDelayRunnable.run(MonoDelay.java:117) ~[reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) [reactor-core-3.3.2.RELEASE.jar!/:3.3.2.RELEASE]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_212]
Caused by: java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 30000ms in 'peek' (and no fallback has been configured)
... 14 common frames omitted

and this is the connection i'm configuring:
```    private ConnectionFactory createConnFactory(BrandSqlConnectionDescriptor brandSqlConnectionDescriptor) {

        ConnectionPoolConfiguration configuration = ConnectionPoolConfiguration.builder(ConnectionFactories.get(builder()
                .option(DRIVER, "pool")
                .option(PROTOCOL, MYSQL_DRIVER)
                .option(HOST, brandSqlConnectionDescriptor.getHost())
                .option(PORT, 3306)
                .option(USER, brandSqlConnectionDescriptor.getUsername())
                .option(PASSWORD,
                        cryptoUtilService.decrypt(brandSqlConnectionDescriptor.getPassword()))
                .option(DATABASE, brandSqlConnectionDescriptor.getSchemaName())
                .build()))
                .maxIdleTime(Duration.ofMinutes(NumberUtils.toInt(confEnvironment.getProperty("MaximumIdleTime"), 30)))
                .initialSize(NumberUtils.toInt(confEnvironment.getProperty("MinimumIdle"), 2))
                .maxSize(NumberUtils.toInt(confEnvironment.getProperty("MaximumPoolSize"), 10))
                .maxAcquireTime(Duration.ofSeconds(NumberUtils.toInt(confEnvironment.getProperty("MaxPoolAquire"), 30)))
                .maxLifeTime(Duration.ofMinutes(NumberUtils.toInt(confEnvironment.getProperty("MaxPoolAquire"), 30)))
                .maxCreateConnectionTime(Duration.ofSeconds(NumberUtils.toInt(confEnvironment.getProperty("MaxConnectionTime"), 30)))
                .acquireRetry((NumberUtils.toInt(confEnvironment.getProperty("MaxRetries"), 3)))
                .validationQuery("SELECT 1")
                .build();
        return new ConnectionPool(configuration);
mp911de commented 4 years ago

This issue is related either to the pool or the driver implementation. Can you file this ticket in https://github.com/r2dbc/r2dbc-pool? /cc @mirromutth