search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.78k stars 2.68k forks source link

[BUG] Reactive MSSQL client randomly loses connection on large queries. #40885

Open SGombert opened 5 months ago

SGombert commented 5 months ago

Describe the bug

Hello everyone,

I want to file a bug report on the reactive MS-SQL client. We run a fully reactive Quarkus app on Azure using Azure SQL as a database. We used Quarkus 3.10.2 and implemented all database-related logic using Panache Reactive and Mutiny. For the most part, everything works as expected. However, for some queries that return a large set of results, the connection to the database closes randomly. This does not always happen for these queries, but it does most of the time.

We then get one of two exceptions: either 

2024-05-29 16:56:35,949 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (vert.x-eventloop-thread-0) HTTP Request to /admin/api/dashboard/attempts/survey-results failed, error id: f80705bf-1495-4015-9438-97c8a508dc19-46: org.hibernate.HibernateException: java.net.SocketException: Connection reset
        at org.hibernate.reactive.sql.results.internal.ReactiveDeferredResultSetAccess.convertException(ReactiveDeferredResultSetAccess.java:229)
        at org.hibernate.reactive.sql.results.internal.ReactiveDeferredResultSetAccess.lambda$executeQuery$2(ReactiveDeferredResultSetAccess.java:176)
        at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187)
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309)
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:144)
        at org.hibernate.reactive.sql.results.internal.ReactiveDeferredResultSetAccess.lambda$executeQuery$3(ReactiveDeferredResultSetAccess.java:175)
        at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187)
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309)
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:144)
        at org.hibernate.reactive.sql.results.internal.ReactiveDeferredResultSetAccess.executeQuery(ReactiveDeferredResultSetAccess.java:151)
        at org.hibernate.reactive.sql.results.internal.ReactiveDeferredResultSetAccess.getReactiveResultSet(ReactiveDeferredResultSetAccess.java:78)
        at org.hibernate.reactive.sql.exec.spi.ReactiveValuesResultSet.lambda$processNext$1(ReactiveValuesResultSet.java:89)
        at org.hibernate.reactive.sql.exec.spi.ReactiveValuesResultSet.advance(ReactiveValuesResultSet.java:129)
        at org.hibernate.reactive.sql.exec.spi.ReactiveValuesResultSet.processNext(ReactiveValuesResultSet.java:88)
        at org.hibernate.reactive.sql.exec.spi.ReactiveValuesResultSet.next(ReactiveValuesResultSet.java:78)
        at org.hibernate.reactive.sql.exec.spi.ReactiveRowProcessingState.next(ReactiveRowProcessingState.java:56)
        at org.hibernate.reactive.sql.results.spi.ReactiveListResultsConsumer.lambda$consume$2(ReactiveListResultsConsumer.java:106)
        at org.hibernate.reactive.util.async.impl.AsyncTrampoline.lambda$asyncWhile$1(AsyncTrampoline.java:215)
        at org.hibernate.reactive.util.async.impl.AsyncTrampoline$TrampolineInternal.unroll(AsyncTrampoline.java:121)
        at org.hibernate.reactive.util.async.impl.AsyncTrampoline$TrampolineInternal.trampoline(AsyncTrampoline.java:102)
        at org.hibernate.reactive.util.async.impl.AsyncTrampoline.asyncWhile(AsyncTrampoline.java:197)
        at org.hibernate.reactive.util.async.impl.AsyncTrampoline.asyncWhile(AsyncTrampoline.java:215)
        at org.hibernate.reactive.util.impl.CompletionStages.whileLoop(CompletionStages.java:424)
        at org.hibernate.reactive.sql.results.spi.ReactiveListResultsConsumer.consume(ReactiveListResultsConsumer.java:106)
        at org.hibernate.reactive.sql.exec.internal.StandardReactiveSelectExecutor.lambda$doExecuteQuery$4(StandardReactiveSelectExecutor.java:204)
        at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187)
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309)
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:144)
        at org.hibernate.reactive.sql.exec.internal.StandardReactiveSelectExecutor.doExecuteQuery(StandardReactiveSelectExecutor.java:148)
        at org.hibernate.reactive.sql.exec.internal.StandardReactiveSelectExecutor.executeQuery(StandardReactiveSelectExecutor.java:118)
        at org.hibernate.reactive.sql.exec.internal.StandardReactiveSelectExecutor.list(StandardReactiveSelectExecutor.java:86)
        at org.hibernate.reactive.sql.exec.internal.StandardReactiveSelectExecutor.list(StandardReactiveSelectExecutor.java:76)
        at org.hibernate.reactive.query.sqm.internal.ConcreteSqmSelectReactiveQueryPlan.lambda$listInterpreter$2(ConcreteSqmSelectReactiveQueryPlan.java:102)
        at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187)
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309)
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:144)
        at org.hibernate.reactive.query.sqm.internal.ConcreteSqmSelectReactiveQueryPlan.lambda$listInterpreter$3(ConcreteSqmSelectReactiveQueryPlan.java:101)
        at java.base/java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1187)
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2309)
        at java.base/java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:144)
        at org.hibernate.reactive.query.sqm.internal.ConcreteSqmSelectReactiveQueryPlan.listInterpreter(ConcreteSqmSelectReactiveQueryPlan.java:99)
        at org.hibernate.reactive.query.sqm.internal.ConcreteSqmSelectReactiveQueryPlan.lambda$new$0(ConcreteSqmSelectReactiveQueryPlan.java:82)
        at org.hibernate.reactive.query.sqm.internal.ConcreteSqmSelectReactiveQueryPlan.withCacheableSqmInterpretation(ConcreteSqmSelectReactiveQueryPlan.java:165)
        at org.hibernate.reactive.query.sqm.internal.ConcreteSqmSelectReactiveQueryPlan.reactivePerformList(ConcreteSqmSelectReactiveQueryPlan.java:122)
        at org.hibernate.reactive.query.sqm.internal.ReactiveQuerySqmImpl.doReactiveList(ReactiveQuerySqmImpl.java:200)
        at org.hibernate.reactive.query.spi.ReactiveAbstractSelectionQuery.doReactiveList(ReactiveAbstractSelectionQuery.java:295)
        at org.hibernate.reactive.query.spi.ReactiveAbstractSelectionQuery.reactiveList(ReactiveAbstractSelectionQuery.java:184)
        at org.hibernate.reactive.query.sqm.internal.ReactiveQuerySqmImpl.reactiveList(ReactiveQuerySqmImpl.java:154)
        at org.hibernate.reactive.query.ReactiveSelectionQuery.getReactiveResultList(ReactiveSelectionQuery.java:43)
        at io.smallrye.context.impl.wrappers.SlowContextualSupplier.get(SlowContextualSupplier.java:21)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage.subscribe(UniCreateFromCompletionStage.java:24)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.lambda$subscribe$0(UniRunSubscribeOn.java:27)
        at org.hibernate.reactive.context.impl.VertxContext.execute(VertxContext.java:91)
        at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.subscribe(UniRunSubscribeOn.java:25)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.performInnerSubscription(UniOnItemTransformToUni.java:81)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.onItem(UniOnItemTransformToUni.java:57)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem$KnownItemSubscription.forward(UniCreateFromKnownItem.java:38)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem.subscribe(UniCreateFromKnownItem.java:23)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni.subscribe(UniOnItemTransformToUni.java:25)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniRetryAtMost$UniRetryAtMostProcessor.onFailure(UniRetryAtMost.java:74)
        at io.smallrye.mutiny.operators.uni.UniOperatorProcessor.onFailure(UniOperatorProcessor.java:55)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forwardResult(UniCreateFromCompletionStage.java:58)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:887)
        at java.base/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2325)
        at java.base/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:144)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forward(UniCreateFromCompletionStage.java:51)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage.subscribe(UniCreateFromCompletionStage.java:35)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.lambda$subscribe$0(UniRunSubscribeOn.java:27)
        at org.hibernate.reactive.context.impl.VertxContext.execute(VertxContext.java:91)
        at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.subscribe(UniRunSubscribeOn.java:25)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.performInnerSubscription(UniOnItemTransformToUni.java:81)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.onItem(UniOnItemTransformToUni.java:57)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem$KnownItemSubscription.forward(UniCreateFromKnownItem.java:38)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem.subscribe(UniCreateFromKnownItem.java:23)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni.subscribe(UniOnItemTransformToUni.java:25)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniRetryAtMost$UniRetryAtMostProcessor.onFailure(UniRetryAtMost.java:74)
        at io.smallrye.mutiny.operators.uni.UniOperatorProcessor.onFailure(UniOperatorProcessor.java:55)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forwardResult(UniCreateFromCompletionStage.java:58)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:887)
        at java.base/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2325)
        at java.base/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:144)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forward(UniCreateFromCompletionStage.java:51)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage.subscribe(UniCreateFromCompletionStage.java:35)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.lambda$subscribe$0(UniRunSubscribeOn.java:27)
        at org.hibernate.reactive.context.impl.VertxContext.execute(VertxContext.java:91)
        at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.subscribe(UniRunSubscribeOn.java:25)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.performInnerSubscription(UniOnItemTransformToUni.java:81)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.onItem(UniOnItemTransformToUni.java:57)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem$KnownItemSubscription.forward(UniCreateFromKnownItem.java:38)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem.subscribe(UniCreateFromKnownItem.java:23)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni.subscribe(UniOnItemTransformToUni.java:25)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniRetryAtMost$UniRetryAtMostProcessor.onFailure(UniRetryAtMost.java:74)
        at io.smallrye.mutiny.operators.uni.UniOperatorProcessor.onFailure(UniOperatorProcessor.java:55)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forwardResult(UniCreateFromCompletionStage.java:58)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:887)
        at java.base/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2325)
        at java.base/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:144)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forward(UniCreateFromCompletionStage.java:51)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage.subscribe(UniCreateFromCompletionStage.java:35)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.lambda$subscribe$0(UniRunSubscribeOn.java:27)
        at org.hibernate.reactive.context.impl.VertxContext.execute(VertxContext.java:91)
        at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.subscribe(UniRunSubscribeOn.java:25)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.performInnerSubscription(UniOnItemTransformToUni.java:81)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.onItem(UniOnItemTransformToUni.java:57)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem$KnownItemSubscription.forward(UniCreateFromKnownItem.java:38)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem.subscribe(UniCreateFromKnownItem.java:23)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni.subscribe(UniOnItemTransformToUni.java:25)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniRetryAtMost$UniRetryAtMostProcessor.onFailure(UniRetryAtMost.java:74)
        at io.smallrye.mutiny.operators.uni.UniOperatorProcessor.onFailure(UniOperatorProcessor.java:55)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forwardResult(UniCreateFromCompletionStage.java:58)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:887)
        at java.base/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2325)
        at java.base/java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:144)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forward(UniCreateFromCompletionStage.java:51)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage.subscribe(UniCreateFromCompletionStage.java:35)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.lambda$subscribe$0(UniRunSubscribeOn.java:27)
        at org.hibernate.reactive.context.impl.VertxContext.execute(VertxContext.java:91)
        at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.subscribe(UniRunSubscribeOn.java:25)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.performInnerSubscription(UniOnItemTransformToUni.java:81)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.onItem(UniOnItemTransformToUni.java:57)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem$KnownItemSubscription.forward(UniCreateFromKnownItem.java:38)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem.subscribe(UniCreateFromKnownItem.java:23)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni.subscribe(UniOnItemTransformToUni.java:25)
        at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
        at io.smallrye.mutiny.operators.uni.UniRetryAtMost$UniRetryAtMostProcessor.onFailure(UniRetryAtMost.java:74)
        at io.smallrye.mutiny.operators.uni.UniOperatorProcessor.onFailure(UniOperatorProcessor.java:55)
        at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage$CompletionStageUniSubscription.forwardResult(UniCreateFromCompletionStage.java:58)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
        at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
        at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
        at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
        at org.hibernate.reactive.util.async.impl.AsyncTrampoline$TrampolineInternal.lambda$unroll$0(AsyncTrampoline.java:123)
        at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
        at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
        at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
        at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
        at io.vertx.core.Future.lambda$toCompletionStage$3(Future.java:583)
        at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
        at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
        at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
        at io.vertx.sqlclient.impl.QueryResultBuilder.tryFail(QueryResultBuilder.java:94)
        at io.vertx.core.Promise.fail(Promise.java:89)
        at io.vertx.core.Promise.handle(Promise.java:53)
        at io.vertx.core.Promise.handle(Promise.java:29)
        at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
        at io.vertx.core.impl.future.FutureBase.lambda$emitFailure$1(FutureBase.java:75)
        at io.vertx.core.impl.ContextImpl.execute(ContextImpl.java:298)
        at io.vertx.core.impl.DuplicatedContext.execute(DuplicatedContext.java:171)
        at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:72)
        at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
        at io.vertx.core.impl.future.PromiseImpl.onFailure(PromiseImpl.java:54)
        at io.vertx.core.impl.future.PromiseImpl.handle(PromiseImpl.java:43)
        at io.vertx.core.impl.future.PromiseImpl.handle(PromiseImpl.java:23)
        at io.vertx.sqlclient.impl.command.CommandResponse.fire(CommandResponse.java:46)
        at io.vertx.sqlclient.impl.SocketConnectionBase.handleMessage(SocketConnectionBase.java:324)
        at io.vertx.mssqlclient.impl.MSSQLSocketConnection.handleMessage(MSSQLSocketConnection.java:164)
        at io.vertx.sqlclient.impl.SocketConnectionBase.lambda$init$0(SocketConnectionBase.java:137)
        at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:328)
        at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:321)
        at io.vertx.core.net.impl.NetSocketImpl.handleMessage(NetSocketImpl.java:388)
        at io.vertx.core.net.impl.ConnectionBase.read(ConnectionBase.java:159)
        at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:153)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
        at io.vertx.mssqlclient.impl.codec.TdsMessageCodec.fail(TdsMessageCodec.java:71)
        at io.vertx.mssqlclient.impl.codec.TdsMessageCodec.fail(TdsMessageCodec.java:63)
        at io.vertx.mssqlclient.impl.codec.TdsMessageCodec.exceptionCaught(TdsMessageCodec.java:53)
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
        at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
        at io.netty.handler.ssl.SslHandler.exceptionCaught(SslHandler.java:1204)
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
        at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377)
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
        at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:125)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:177)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.net.SocketException: Connection reset
        at java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:394)
        at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:426)
        at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:255)
        at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132)
        at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:357)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151)
        ... 8 more

or 

2024-05-29 16:56:35,341 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (vert.x-eventloop-thread-0) HTTP Request to /admin/api/dashboard/attempts/survey-results failed, error id: f80705bf-1495-4015-9438-97c8a508dc19-45: org.hibernate.HibernateException: io.vertx.core.impl.NoStackTraceThrowable: Connection is not active now, current status: CLOSED
        at org.hibernate.reactive.sql.results.internal.ReactiveDeferredResultSetAccess.convertException(ReactiveDeferredResultSetAccess.java:229)
        at org.hibernate.reactive.sql.results.internal.ReactiveDeferredResultSetAccess.lambda$executeQuery$2(ReactiveDeferredResultSetAccess.java:176)
        at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1150)
        at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
        at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
        at io.vertx.core.Future.lambda$toCompletionStage$3(Future.java:583)
        at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
        at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:81)
        at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:278)
        at io.vertx.sqlclient.impl.QueryResultBuilder.tryFail(QueryResultBuilder.java:94)
        at io.vertx.core.Promise.fail(Promise.java:89)
        at io.vertx.core.Promise.handle(Promise.java:53)
        at io.vertx.core.Promise.handle(Promise.java:29)
        at io.vertx.core.impl.future.FutureImpl$4.onFailure(FutureImpl.java:188)
        at io.vertx.core.impl.future.FutureBase.lambda$emitFailure$1(FutureBase.java:75)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: io.vertx.core.impl.NoStackTraceThrowable: Connection is not active now, current status: CLOSED

The problem occurred much more often in the beginning, but most previous occurrences could be solved by setting the following config parameter: quarkus.datasource.reactive.mssql.packet-size=8000

Nonetheless, it still happens from time to time. I basically played around with all the config parameters and have no clue what to do. I guess it is a bug in the framework.

These are my other db-related config parameters:

quarkus.datasource.reactive.idle-timeout=PT30M
quarkus.datasource.reactive.max-lifetime=PT60M
hibernate.vertx.pool.idle_timeout=PT30M
quarkus.datasource.reactive.max-size = 30
quarkus.datasource.reactive.cache-prepared-statements = true
quarkus.datasource.reactive.mssql.packet-size=8000
quarkus.datasource.reactive.mssql.ssl=true
quarkus.hibernate-orm.database.generation=update
quarkus.datasource.db-kind=mssql

Expected behavior

The expected behaviour should be that my query executes without any problem deterministically.

The object which we want to get out of the DB with the query (happens with other objects as well, though):

package science.edutec.neobridge.entities.survey;

import org.hibernate.annotations.Cache;
import org.hibernate.annotations.CacheConcurrencyStrategy;

import io.quarkus.hibernate.reactive.panache.PanacheEntity;

import jakarta.persistence.Entity;
import jakarta.persistence.FetchType;
import jakarta.persistence.Index;
import jakarta.persistence.JoinColumn;
import jakarta.persistence.Lob;
import jakarta.persistence.ManyToOne;
import jakarta.persistence.Table;
import science.edutec.neobridge.entities.meta.AssessmentPhaseInSession;

@Entity
@Table(indexes = {
    @Index(name = "byApisId", columnList = "phase_in_session_id")
})
public class ExternalSurveyAnswer extends PanacheEntity {
    @Lob
    public String content;

    @ManyToOne(fetch = FetchType.EAGER)
    @JoinColumn(name="question_id")
    @Cache(usage = CacheConcurrencyStrategy.READ_WRITE)
    public ExternalSurveyQuestion surveyQuestion;

    @ManyToOne(fetch = FetchType.EAGER)
    @JoinColumn(name="phase_in_session_id")
    @Cache(usage = CacheConcurrencyStrategy.READ_WRITE)
    public AssessmentPhaseInSession phase;
}

Actual behavior

As seen in the bug description.

How to Reproduce?

It is hard to reproduce because it does not occur deterministically. However, it continually occurs for very large queries. To reproduce it, fill a table in AzureSQL with around 300k rows, then query around 20k of them.

Output of uname -a or ver

No response

Output of java -version

java version "17.0.8" 2023-07-18 LTS Java(TM) SE Runtime Environment (build 17.0.8+9-LTS-211) Java HotSpot(TM) 64-Bit Server VM (build 17.0.8+9-LTS-211, mixed mode, sharing)

Quarkus version or git rev

3.11

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response

quarkus-bot[bot] commented 5 months ago

/cc @tsegismont (reactive-sql-clients), @vietj (reactive-sql-clients)

tsegismont commented 5 months ago

Hi @SGombert

It seems the application uses SSL, in this case the problem could be related to https://github.com/eclipse-vertx/vertx-sql-client/issues/1433

This bug has been resolved in Vert.x 4.5.8. Any chance for you to try this version?

SGombert commented 5 months ago

Hi, thanks @tsegismont

I will try it and then report if it helped.

tsegismont commented 3 months ago

@SGombert have you been able to try with the fixed version?