search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
1.19k stars 24.85k forks source link

[CI] CCSDuelIT testTermsLookup failing #94269

Closed slobodanadamovic closed 1 year ago

slobodanadamovic commented 1 year ago

Build scan: https://gradle-enterprise.elastic.co/s/5a257lvuesvca/tests/:qa:multi-cluster-search:v8.8.0%23multi-cluster/org.elasticsearch.search.CCSDuelIT/testTermsLookup

Reproduction line:

./gradlew ':qa:multi-cluster-search:v8.8.0#multi-cluster' -Dtests.class="org.elasticsearch.search.CCSDuelIT" -Dtests.method="testTermsLookup" -Dtests.seed=749CFADD9B578FC -Dtests.bwc=true -Dtests.locale=ga -Dtests.timezone=Asia/Chita -Druntime.java=18

Applicable branches: main

Reproduces locally?: Didn't try

Failure history: https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.search.CCSDuelIT&tests.test=testTermsLookup

Failure excerpt:

java.lang.AssertionError: both requests returned an exception

  at __randomizedtesting.SeedInfo.seed([749CFADD9B578FC:C6DBFF4B6CBA29C6]:0)
  at org.elasticsearch.search.CCSDuelIT.duelSearch(CCSDuelIT.java:782)
  at org.elasticsearch.search.CCSDuelIT.testTermsLookup(CCSDuelIT.java:668)
  at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
  at java.lang.reflect.Method.invoke(Method.java:577)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  at java.lang.Thread.run(Thread.java:833)

  Caused by: org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=connect_transport_exception, reason=[][[::1]:37153] connect_exception]

    at org.elasticsearch.rest.RestResponse.errorFromXContent(RestResponse.java:213)
    at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:642)
    at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:619)
    at org.elasticsearch.client.RestHighLevelClient$1.onFailure(RestHighLevelClient.java:593)
    at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:684)
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:409)
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:393)
    at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
    at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182)
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40)
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
    at java.lang.Thread.run(Thread.java:833)

    Caused by: org.elasticsearch.ElasticsearchException: Elasticsearch exception [type=uncategorized_execution_exception, reason=Failed execution]

      at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:492)
      at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:403)
      at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:433)
      at org.elasticsearch.ElasticsearchException.failureFromXContent(ElasticsearchException.java:599)
      at org.elasticsearch.rest.RestResponse.errorFromXContent(RestResponse.java:205)
      at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:642)
      at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:619)
      at org.elasticsearch.client.RestHighLevelClient$1.onFailure(RestHighLevelClient.java:593)
      at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:684)
      at org.elasticsearch.client.RestClient$1.completed(RestClient.java:409)
      at org.elasticsearch.client.RestClient$1.completed(RestClient.java:393)
      at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
      at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182)
      at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
      at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
      at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
      at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87)
      at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40)
      at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
      at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
      at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
      at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
      at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
      at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
      at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
      at java.lang.Thread.run(Thread.java:833)

      Caused by: org.elasticsearch.ElasticsearchException: Elasticsearch exception [type=execution_exception, reason=io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /[0:0:0:0:0:0:0:1]:37153]

        at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:492)
        at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:403)
        at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:433)
        at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:403)
        at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:433)
        at org.elasticsearch.ElasticsearchException.failureFromXContent(ElasticsearchException.java:599)
        at org.elasticsearch.rest.RestResponse.errorFromXContent(RestResponse.java:205)
        at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:642)
        at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:619)
        at org.elasticsearch.client.RestHighLevelClient$1.onFailure(RestHighLevelClient.java:593)
        at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:684)
        at org.elasticsearch.client.RestClient$1.completed(RestClient.java:409)
        at org.elasticsearch.client.RestClient$1.completed(RestClient.java:393)
        at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
        at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182)
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
        at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87)
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40)
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
        at java.lang.Thread.run(Thread.java:833)

        Caused by: org.elasticsearch.ElasticsearchException: Elasticsearch exception [type=annotated_connect_exception, reason=Connection refused: /[0:0:0:0:0:0:0:1]:37153]

          at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:492)
          at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:403)
          at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:433)
          at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:403)
          at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:433)
          at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:403)
          at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:433)
          at org.elasticsearch.ElasticsearchException.failureFromXContent(ElasticsearchException.java:599)
          at org.elasticsearch.rest.RestResponse.errorFromXContent(RestResponse.java:205)
          at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:642)
          at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:619)
          at org.elasticsearch.client.RestHighLevelClient$1.onFailure(RestHighLevelClient.java:593)
          at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:684)
          at org.elasticsearch.client.RestClient$1.completed(RestClient.java:409)
          at org.elasticsearch.client.RestClient$1.completed(RestClient.java:393)
          at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
          at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182)
          at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
          at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
          at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
          at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87)
          at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40)
          at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
          at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
          at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
          at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
          at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
          at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
          at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
          at java.lang.Thread.run(Thread.java:833)

          Caused by: org.elasticsearch.ElasticsearchException: Elasticsearch exception [type=connect_exception, reason=Connection refused]

            at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:492)
            at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:403)
            at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:433)
            at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:403)
            at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:433)
            at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:403)
            at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:433)
            at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:403)
            at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:433)
            at org.elasticsearch.ElasticsearchException.failureFromXContent(ElasticsearchException.java:599)
            at org.elasticsearch.rest.RestResponse.errorFromXContent(RestResponse.java:205)
            at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:642)
            at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:619)
            at org.elasticsearch.client.RestHighLevelClient$1.onFailure(RestHighLevelClient.java:593)
            at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:684)
            at org.elasticsearch.client.RestClient$1.completed(RestClient.java:409)
            at org.elasticsearch.client.RestClient$1.completed(RestClient.java:393)
            at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
            at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182)
            at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
            at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
            at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
            at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87)
            at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40)
            at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
            at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
            at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
            at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
            at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
            at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
            at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
            at java.lang.Thread.run(Thread.java:833)
elasticsearchmachine commented 1 year ago

Pinging @elastic/es-distributed (Team:Distributed)

slobodanadamovic commented 1 year ago

The actual cause is:

[2023-03-02T08:51:02,724][WARN ][o.e.t.OutboundHandler    ] [v8.8.0-remote-0] failed to serialize outbound message [Response{103}{false}{false}{false}{class org.elasticsearch.transport.TransportActionProxy$BytesTransportResponse}]
java.lang.UnsupportedOperationException: null
    at org.elasticsearch.common.bytes.BytesReference.array(BytesReference.java:203) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.common.bytes.ReleasableBytesReference.array(ReleasableBytesReference.java:239) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.TransportActionProxy$BytesTransportResponse.writeTo(TransportActionProxy.java:113) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.OutboundMessage.serialize(OutboundMessage.java:70) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.OutboundHandler.sendMessage(OutboundHandler.java:178) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.OutboundHandler.sendResponse(OutboundHandler.java:138) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.TcpTransportChannel.sendResponse(TcpTransportChannel.java:58) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:44) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.TransportActionProxy$ProxyRequestHandler$1.handleResponse(TransportActionProxy.java:62) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.TransportService$UnregisterChildTransportResponseHandler.handleResponse(TransportService.java:1653) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1372) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.InboundHandler.doHandleResponse(InboundHandler.java:387) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.InboundHandler.handleResponse(InboundHandler.java:336) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:142) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:95) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:832) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:150) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:121) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:86) ~[elasticsearch-8.8.0-SNAPSHOT.jar:?]
    at org.elasticsearch.transport.netty4.Netty4MessageInboundHandler.channelRead(Netty4MessageInboundHandler.java:63) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
    at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[?:?]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[?:?]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) ~[?:?]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) ~[?:?]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[?:?]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[?:?]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
    at java.lang.Thread.run(Thread.java:833) ~[?:?]

Which seems to be introduced by PR: https://github.com/elastic/elasticsearch/pull/93799 cc: @dnhatn

original-brownbear commented 1 year ago

Sec fixing this, I know where this is coming from.

dnhatn commented 1 year ago

@original-brownbear Thanks for fixing this quickly.