search

quarkiverse / quarkus-cxf

Quarkus CXF Extension to support SOAP based web services.
Apache License 2.0
79 stars 60 forks source link

VertxHttpClientHTTPConduit-backed client hangs after upgrading to Quarkus CXF 3.16.0/3.16.1 #1582

Closed argenstijn closed 1 day ago

argenstijn commented 3 weeks ago

I experience some problem using 3.16.X version when using the default http-conduit-factory. When using the 'quarkus.cxf.http-conduit-factory=URLConnectionHTTPConduitFactory' all seems to work again.

  1. In the old situation before 3.16.X the header 'Content-Type' contains a charset=UTF-8. In the current version not anymore!
-----------------------------------------------------------------------------------------------------------------------
| Closest stub                                             | Request                                                  |
-----------------------------------------------------------------------------------------------------------------------
                                                             |
POST                                                     | POST
/xxxx/xxxxxxx                                         |    /xxxx/xxxxxxx
                                                            |
Content-Type: text/xml; charset=UTF-8                      | Content-Type: text/xml                              <<<<< Header does not match

Edit by @ppalaga: I created a separate issue https://github.com/quarkiverse/quarkus-cxf/issues/1599 for the above

  1. in the new situation the thread seems to hang using VertxHttpClientHTTPConduitFactory
2024-11-05 09:24:15,753 WARN  [org.apa.cxf.pha.PhaseInterceptorChain] (executor-thread-1) Interceptor for {http://www.quinity.com/qis/soap/qispartyservice}QisPartyService#{http://www.quinity.com/qis/soap/qispartyservice}addUpdateParties has thrown exception, unwinding now: org.apache.cxf.interceptor.Fault: Could not send Message.
    at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:67)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
    at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:530)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:441)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:356)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:314)
    at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:140)
    at jdk.proxy5/jdk.proxy5.$Proxy262.addUpdateParties(Unknown Source)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore.createOrUpdate(CustomersStore.java:65)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_Subclass.createOrUpdate$$superforward(Unknown Source)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_Subclass$$function$$1.apply(Unknown Source)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor.lambda$syncFlow$3(FaultToleranceInterceptor.java:267)
    at io.smallrye.faulttolerance.core.InvocationContext.call(InvocationContext.java:19)
    at io.smallrye.faulttolerance.core.Invocation.apply(Invocation.java:29)
    at io.smallrye.faulttolerance.core.retry.Retry.doApply(Retry.java:104)
    at io.smallrye.faulttolerance.core.retry.Retry.apply(Retry.java:50)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor.syncFlow(FaultToleranceInterceptor.java:269)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor.intercept(FaultToleranceInterceptor.java:196)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor_Bean.intercept(Unknown Source)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
    at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_Subclass.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_ClientProxy.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource.createOrUpdate(CustomersResource.java:57)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_Subclass.createOrUpdate$$superforward(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_Subclass$$function$$3.apply(Unknown Source)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
    at io.quarkus.hibernate.validator.runtime.interceptor.AbstractMethodValidationInterceptor.validateMethodInvocation(AbstractMethodValidationInterceptor.java:71)
    at io.quarkus.hibernate.validator.runtime.jaxrs.ResteasyReactiveEndPointValidationInterceptor.validateMethodInvocation(ResteasyReactiveEndPointValidationInterceptor.java:21)
    at io.quarkus.hibernate.validator.runtime.jaxrs.ResteasyReactiveEndPointValidationInterceptor_Bean.intercept(Unknown Source)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
    at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_Subclass.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_ClientProxy.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource$quarkusrestinvoker$createOrUpdate_f481820868dbc58995edff6a410c8d38b154d171.invoke(Unknown Source)
    at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
    at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:141)
    at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
    at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:627)
    at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2675)
    at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2654)
    at org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1627)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1594)
    at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
    at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: org.apache.cxf.transport.http.HTTPException: HTTP response '404: Not Found' when communicating with http://localhost:9888//xxxx/xxxxxxx  
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$ResponseHandler.doProcessResponseCode(VertxHttpClientHTTPConduit.java:908)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$ResponseHandler.handle(VertxHttpClientHTTPConduit.java:807)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$ResponseHandler.handle(VertxHttpClientHTTPConduit.java:786)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyHandler.handle(VertxHttpClientHTTPConduit.java:522)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyHandler.handle(VertxHttpClientHTTPConduit.java:374)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyOutputStream.close(VertxHttpClientHTTPConduit.java:369)
    at org.apache.cxf.ext.logging.LoggingOutputStream.postClose(LoggingOutputStream.java:53)
    at org.apache.cxf.io.CachedOutputStream.close(CachedOutputStream.java:228)
    at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
    at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:717)
    at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:63)
    ... 52 more

2024-11-05 09:24:15,755 INFO  [com.nn.aav.ins.bus.cus.con.pol.CustomersStore] (executor-thread-1) createOrUpdate invoked for customer with id -213131223
2024-11-05 09:24:19,561 WARN  [io.ver.cor.imp.BlockedThreadChecker] (vertx-blocked-thread-checker) Thread Thread[vert.x-eventloop-thread-2,5,main] has been blocked for 3809 ms, time limit is 2000 ms: io.vertx.core.VertxException: Thread blocked
    at java.base/java.lang.Object.wait0(Native Method)
    at java.base/java.lang.Object.wait(Object.java:366)
    at java.base/java.io.PipedInputStream.awaitSpace(PipedInputStream.java:279)
    at java.base/java.io.PipedInputStream.receive(PipedInputStream.java:237)
    at java.base/java.io.PipedOutputStream.write(PipedOutputStream.java:154)
    at java.base/java.io.OutputStream.write(OutputStream.java:124)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyHandler.lambda$pipe$3(VertxHttpClientHTTPConduit.java:694)
    at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:270)
    at io.vertx.core.http.impl.HttpEventHandler.handleChunk(HttpEventHandler.java:51)
    at io.vertx.core.http.impl.HttpClientResponseImpl.handleChunk(HttpClientResponseImpl.java:239)
    at io.vertx.core.http.impl.Http1xClientConnection$StreamImpl.lambda$new$0(Http1xClientConnection.java:429)
    at io.vertx.core.streams.impl.InboundBuffer.handleEvent(InboundBuffer.java:279)
    at io.vertx.core.streams.impl.InboundBuffer.write(InboundBuffer.java:157)
    at io.vertx.core.http.impl.Http1xClientConnection$StreamImpl.handleChunk(Http1xClientConnection.java:705)
    at io.vertx.core.impl.ContextImpl.execute(ContextImpl.java:313)
    at io.vertx.core.impl.DuplicatedContext.execute(DuplicatedContext.java:159)
    at io.vertx.core.http.impl.Http1xClientConnection.handleResponseChunk(Http1xClientConnection.java:888)
    at io.vertx.core.http.impl.Http1xClientConnection.handleHttpMessage(Http1xClientConnection.java:807)
    at io.vertx.core.http.impl.Http1xClientConnection.handleMessage(Http1xClientConnection.java:774)
    at io.vertx.core.net.impl.ConnectionBase.read(ConnectionBase.java:159)
    at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:153)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:1583)

2024-11-05 09:24:20,771 WARN  [org.apa.cxf.pha.PhaseInterceptorChain] (executor-thread-1) Interceptor for {http://www.quinity.com/qis/soap/qispartyservice}QisPartyService#{http://www.quinity.com/qis/soap/qispartyservice}addUpdateParties has thrown exception, unwinding now: org.apache.cxf.binding.soap.SoapFault: Error writing to XMLStreamWriter.
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:318)
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:294)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
    at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:530)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:441)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:356)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:314)
    at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:140)
    at jdk.proxy5/jdk.proxy5.$Proxy262.addUpdateParties(Unknown Source)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore.createOrUpdate(CustomersStore.java:65)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_Subclass.createOrUpdate$$superforward(Unknown Source)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_Subclass$$function$$1.apply(Unknown Source)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor.lambda$syncFlow$3(FaultToleranceInterceptor.java:267)
    at io.smallrye.faulttolerance.core.InvocationContext.call(InvocationContext.java:19)
    at io.smallrye.faulttolerance.core.Invocation.apply(Invocation.java:29)
    at io.smallrye.faulttolerance.core.retry.Retry.doApply(Retry.java:104)
    at io.smallrye.faulttolerance.core.retry.Retry.apply(Retry.java:50)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor.syncFlow(FaultToleranceInterceptor.java:269)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor.intercept(FaultToleranceInterceptor.java:196)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor_Bean.intercept(Unknown Source)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
    at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_Subclass.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_ClientProxy.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource.createOrUpdate(CustomersResource.java:57)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_Subclass.createOrUpdate$$superforward(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_Subclass$$function$$3.apply(Unknown Source)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
    at io.quarkus.hibernate.validator.runtime.interceptor.AbstractMethodValidationInterceptor.validateMethodInvocation(AbstractMethodValidationInterceptor.java:71)
    at io.quarkus.hibernate.validator.runtime.jaxrs.ResteasyReactiveEndPointValidationInterceptor.validateMethodInvocation(ResteasyReactiveEndPointValidationInterceptor.java:21)
    at io.quarkus.hibernate.validator.runtime.jaxrs.ResteasyReactiveEndPointValidationInterceptor_Bean.intercept(Unknown Source)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
    at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_Subclass.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_ClientProxy.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource$quarkusrestinvoker$createOrUpdate_f481820868dbc58995edff6a410c8d38b154d171.invoke(Unknown Source)
    at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
    at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:141)
    at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
    at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:627)
    at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2675)
    at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2654)
    at org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1627)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1594)
    at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
    at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: com.ctc.wstx.exc.WstxIOException: Timeout waiting for HTTP connect to http://localhost:9888/Axon/QisPartyService
    at com.ctc.wstx.sw.BaseStreamWriter._finishDocument(BaseStreamWriter.java:1480)
    at com.ctc.wstx.sw.BaseStreamWriter.writeEndDocument(BaseStreamWriter.java:550)
    at org.apache.cxf.staxutils.DelegatingXMLStreamWriter.writeEndDocument(DelegatingXMLStreamWriter.java:119)
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:307)
    ... 53 more
Caused by: java.net.SocketTimeoutException: Timeout waiting for HTTP connect to http://localhost:9888/Axon/QisPartyService
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyHandler.awaitRequest(VertxHttpClientHTTPConduit.java:664)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyHandler.handle(VertxHttpClientHTTPConduit.java:511)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyHandler.handle(VertxHttpClientHTTPConduit.java:374)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyOutputStream.write(VertxHttpClientHTTPConduit.java:336)
    at org.apache.cxf.io.CacheAndWriteOutputStream.write(CacheAndWriteOutputStream.java:81)
    at com.ctc.wstx.io.UTF8Writer.flush(UTF8Writer.java:100)
    at com.ctc.wstx.sw.BufferingXmlWriter.flush(BufferingXmlWriter.java:242)
    at com.ctc.wstx.sw.BufferingXmlWriter.close(BufferingXmlWriter.java:215)
    at com.ctc.wstx.sw.BaseStreamWriter._finishDocument(BaseStreamWriter.java:1478)
    ... 56 more

2024-11-05 09:24:21,055 INFO  [com.nn.aav.ins.bus.cus.con.pol.CustomersStore] (executor-thread-1) createOrUpdate invoked for customer with id -213131223
2024-11-05 09:24:26,076 WARN  [org.apa.cxf.pha.PhaseInterceptorChain] (executor-thread-1) Interceptor for {http://www.quinity.com/qis/soap/qispartyservice}QisPartyService#{http://www.quinity.com/qis/soap/qispartyservice}addUpdateParties has thrown exception, unwinding now: org.apache.cxf.binding.soap.SoapFault: Error writing to XMLStreamWriter.
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:318)
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:294)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
    at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:530)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:441)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:356)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:314)
    at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:140)
    at jdk.proxy5/jdk.proxy5.$Proxy262.addUpdateParties(Unknown Source)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore.createOrUpdate(CustomersStore.java:65)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_Subclass.createOrUpdate$$superforward(Unknown Source)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_Subclass$$function$$1.apply(Unknown Source)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor.lambda$syncFlow$3(FaultToleranceInterceptor.java:267)
    at io.smallrye.faulttolerance.core.InvocationContext.call(InvocationContext.java:19)
    at io.smallrye.faulttolerance.core.Invocation.apply(Invocation.java:29)
    at io.smallrye.faulttolerance.core.retry.Retry.doApply(Retry.java:104)
    at io.smallrye.faulttolerance.core.retry.Retry.apply(Retry.java:50)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor.syncFlow(FaultToleranceInterceptor.java:269)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor.intercept(FaultToleranceInterceptor.java:196)
    at io.smallrye.faulttolerance.FaultToleranceInterceptor_Bean.intercept(Unknown Source)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
    at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_Subclass.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.control.policyadministration.CustomersStore_ClientProxy.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource.createOrUpdate(CustomersResource.java:57)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_Subclass.createOrUpdate$$superforward(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_Subclass$$function$$3.apply(Unknown Source)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
    at io.quarkus.hibernate.validator.runtime.interceptor.AbstractMethodValidationInterceptor.validateMethodInvocation(AbstractMethodValidationInterceptor.java:71)
    at io.quarkus.hibernate.validator.runtime.jaxrs.ResteasyReactiveEndPointValidationInterceptor.validateMethodInvocation(ResteasyReactiveEndPointValidationInterceptor.java:21)
    at io.quarkus.hibernate.validator.runtime.jaxrs.ResteasyReactiveEndPointValidationInterceptor_Bean.intercept(Unknown Source)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
    at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_Subclass.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource_ClientProxy.createOrUpdate(Unknown Source)
    at com.nn.aav.insurance.business.customers.boundary.CustomersResource$quarkusrestinvoker$createOrUpdate_f481820868dbc58995edff6a410c8d38b154d171.invoke(Unknown Source)
    at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
    at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:141)
    at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
    at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:627)
    at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2675)
    at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2654)
    at org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1627)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1594)
    at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
    at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: com.ctc.wstx.exc.WstxIOException: Timeout waiting for HTTP connect to http://localhost:9888/Axon/QisPartyService
    at com.ctc.wstx.sw.BaseStreamWriter._finishDocument(BaseStreamWriter.java:1480)
    at com.ctc.wstx.sw.BaseStreamWriter.writeEndDocument(BaseStreamWriter.java:550)
    at org.apache.cxf.staxutils.DelegatingXMLStreamWriter.writeEndDocument(DelegatingXMLStreamWriter.java:119)
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:307)
    ... 53 more
Caused by: java.net.SocketTimeoutException: Timeout waiting for HTTP connect to http://localhost:9888/Axon/QisPartyService
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyHandler.awaitRequest(VertxHttpClientHTTPConduit.java:664)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyHandler.handle(VertxHttpClientHTTPConduit.java:511)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyHandler.handle(VertxHttpClientHTTPConduit.java:374)
    at io.quarkiverse.cxf.vertx.http.client.VertxHttpClientHTTPConduit$RequestBodyOutputStream.write(VertxHttpClientHTTPConduit.java:336)
    at org.apache.cxf.io.CacheAndWriteOutputStream.write(CacheAndWriteOutputStream.java:81)
    at com.ctc.wstx.io.UTF8Writer.flush(UTF8Writer.java:100)
    at com.ctc.wstx.sw.BufferingXmlWriter.flush(BufferingXmlWriter.java:242)
    at com.ctc.wstx.sw.BufferingXmlWriter.close(BufferingXmlWriter.java:215)
    at com.ctc.wstx.sw.BaseStreamWriter._finishDocument(BaseStreamWriter.java:1478)
    ... 56 more

2024-11-05 09:24:26,648 INFO  [com.nn.aav.ins.bus.cus.con.pol.CustomersStore] (executor-thread-1) createOrUpdate invoked for customer with id -213131223
2024-11-05 09:24:31,664 WARN  [org.apa.cxf.pha.PhaseInterceptorChain] (executor-thread-1) Interceptor for {http://www.quinity.com/qis/soap/qispartyservice}QisPartyService#{http://www.quinity.com/qis/soap/qispartyservice}addUpdateParties has thrown exception, unwinding now: org.apache.cxf.binding.soap.SoapFault: Error writing to XMLStreamWriter.
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:318)
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:294)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
    at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:530)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:441)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:356)
    at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:314)
    at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
    at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:140)
    at jdk.proxy5/jdk.proxy5.$Proxy262.addUpdateParties(Unknown Source)
mcanzerini commented 3 weeks ago

Same for me.

Thanks for the workaround: quarkus.cxf.http-conduit-factory=URLConnectionHTTPConduitFactory

ppalaga commented 3 weeks ago

Thanks for the report @argenstijn! Yes, we switched the default to VertxHttpClientHTTPConduit in 3.16. I'll have a look.

krisztiankocsis commented 2 weeks ago

I can confirm both the issue and the workaround that it helps.

ppalaga commented 1 week ago

@argenstijn could you please provide a bit more context to those stack traces?

(1) Did you expect those calls going through $Proxy262.addUpdateParties() to succeed because you were sure that the remote service was up and running? Could it not be that it was still starting or down for other reasons and the client was properly reporting that the service is not available?

(2) The warning about vert.x-eventloop-thread being blocked is very useful and points at a wrong implementation in Quarkus CXF: the eventloop is pumping the data from the response into a pipe with limited buffer that blocks when the consumer running on a worker thread is not fast enough to swallow the data. We need to fix it.

I wonder whether you have any hints that (1) and (2) have (not) the same root cause?

ppalaga commented 4 days ago

(2) The warning about vert.x-eventloop-thread being blocked is very useful and points at a wrong implementation in Quarkus CXF: the eventloop is pumping the data from the response into a pipe with limited buffer that blocks when the consumer running on a worker thread is not fast enough to swallow the data. We need to fix it.

I created a separate issue https://github.com/quarkiverse/quarkus-cxf/issues/1613 for this part and I have fixed it already.

@argenstijn, I am keeping this issue open, because it is not clear to me whether the fixes for #1613 and https://github.com/quarkiverse/quarkus-cxf/issues/1599 solve all your issues. Unless you provide more feedback I am going to release 3.17.0 in the coming days and let you check whether anything is left unresolved.

argenstijn commented 1 day ago

@ppalaga Sorry for my late response, i was gone for a week or so.

I think you solved the issue. I will check right now if it is fixed in the new version

ppalaga commented 1 day ago

@argenstijn no worries at all. It would still be great to hear whether 3.17.0, that is out since yesterday, has solved all your issues.

argenstijn commented 1 day ago

tested and issue is solved :)