search

jetty / jetty.project

Eclipse Jetty® - Web Container & Clients - supports HTTP/2, HTTP/1.1, HTTP/1.0, websocket, servlets, and more
https://eclipse.dev/jetty
Other
3.86k stars 1.91k forks source link

Unconsumed content exception in the logs #12298

Open iamsanjay opened 1 month ago

iamsanjay commented 1 month ago

jetty_logs.txt Jetty Version 10

Java Version 11

Question Why I am seeing Unconsumed content exception errors in the logs?

I am creating a POST request and sending a data to the Jetty server, and then receiving the response. From the logs, It seems that the server is able to process the stream without any issue. However, while parsing response it is throwing exception -- Unconsumed Content.

Attached the logs!

2854 DEBUG (httpShardExecutor-16-thread-2) [n: c: s: r: x: t:] o.e.j.h.p.Parser Parsed [HEADERS|64|4|1] frame header from java.nio.DirectByteBuffer[pos=9 lim=152 cap=16384]@926c80dd
2854 DEBUG (qtp1987626026-62) [n: c: s: r: x: t:] o.e.j.h.HTTP2Session Closed stream HTTP2Stream@6c9e1ce0#1@28ff38bd{sendWindow=8388538,recvWindow=524079,queue=0,demand=0,reset=false/false,CLOSED,age=345,attachment=ServerHttpChannelOverHTTP2@4f2412f6{s=HttpChannelState@379427b4{s=HANDLING rs=COMPLETING os=COMMITTED is=IDLE awp=false se=false i=false al=0},r=1,c=false/false,a=HANDLING,uri=http://127.0.0.1:58631/solr/admin/cores,age=343}#1} for HTTP2ServerSession@28ff38bd{local:/127.0.0.1:58631,remote:/127.0.0.1:58640,sendWindow=16777146,recvWindow=1048367,state=[streams=1,NOT_CLOSED,goAwayRecv=null,goAwaySent=null,failure=null]}
2854 DEBUG (qtp1987626026-62) [n: c: s: r: x: t:] o.e.j.h.HTTP2Session Destroyed stream #1 for HTTP2ServerSession@28ff38bd{local:/127.0.0.1:58631,remote:/127.0.0.1:58640,sendWindow=16777146,recvWindow=1048367,state=[streams=1,NOT_CLOSED,goAwayRecv=null,goAwaySent=null,failure=null]}
2854 DEBUG (httpShardExecutor-16-thread-2) [n: c: s: r: x: t:] o.e.j.h.p.HeadersBodyParser Parsed HEADERS frame hpack from java.nio.DirectByteBuffer[pos=73 lim=152 cap=16384]
2854 DEBUG (qtp1987626026-62) [n: c: s: r: x: t:] o.e.j.h.s.HttpTransportOverHTTP2 HTTP2 Response #1/28ff38bd commit success
2854 DEBUG (httpShardExecutor-16-thread-2) [n: c: s: r: x: t:] o.e.j.h.c.HTTP2ClientSession Received HeadersFrame@2081e226#1[end=false,{HTTP/2.0{s=200,h=4,cl=70}},priority=null]
2854 DEBUG (qtp1987626026-62) [n: c: s: r: x: t:] o.e.j.h.HTTP2Flusher Flushing HTTP2ServerSession@28ff38bd{local:/127.0.0.1:58631,remote:/127.0.0.1:58640,sendWindow=16777146,recvWindow=1048367,state=[streams=0,NOT_CLOSED,goAwayRecv=null,goAwaySent=null,failure=null]}
2854 DEBUG (httpShardExecutor-16-thread-2) [n: c: s: r: x: t:] o.e.j.h.HTTP2Stream Update close for HTTP2Stream@168a0731#1@55c13bd{sendWindow=524079,recvWindow=8388608,queue=0,demand=0,reset=false/false,LOCALLY_CLOSED,age=348,attachment=HttpReceiverOverHTTP2@274fe30c(rsp=IDLE,failure=null)} update=false event=RECEIVED
2854 DEBUG (qtp1987626026-62) [n: c: s: r: x: t:] o.e.j.h.HTTP2Flusher Flushed HTTP2ServerSession@28ff38bd{local:/127.0.0.1:58631,remote:/127.0.0.1:58640,sendWindow=16777146,recvWindow=1048367,state=[streams=0,NOT_CLOSED,goAwayRecv=null,goAwaySent=null,failure=null]}
2854 DEBUG (qtp1987626026-62) [n: c: s: r: x: t:] o.e.j.h.s.HttpChannelOverHTTP2 failing all content with org.eclipse.jetty.util.StaticException: Unconsumed content ServerHttpChannelOverHTTP2@4f2412f6{s=HttpChannelState@379427b4{s=IDLE rs=COMPLETED os=COMPLETED is=IDLE awp=false se=false i=false al=0},r=1,c=true/true,a=IDLE,uri=http://127.0.0.1:58631/solr/admin/cores,age=343}#1
2855 DEBUG (httpShardExecutor-16-thread-2) [n: c: s: r: x: t:] o.e.j.h.HTTP2Stream Demand 1/1, stalling data processing for HTTP2Stream@168a0731#1@55c13bd{sendWindow=524079,recvWindow=8388608,queue=0,demand=1,reset=false/false,LOCALLY_CLOSED,age=349,attachment=HttpReceiverOverHTTP2@274fe30c(rsp=HEADERS,failure=null)}
2855 DEBUG (httpShardExecutor-16-thread-2) [n: c: s: r: x: t:] o.e.j.h.p.Parser Parsed HEADERS frame body from java.nio.DirectByteBuffer[pos=73 lim=152 cap=16384]@aa95dbed

Basically we are indexing one document and then finally shutdown the server as we shut down we also see this exception.

2913 DEBUG (qtp1987626026-58) [n: c: s: r: x: t:] o.e.j.h.HTTP2Connection Processing session failure on HTTP2ServerSession@51d68840{local:null,remote:null,sendWindow=16777125,recvWindow=1046903,state=[streams=0,CLOSED,goAwayRecv=GoAwayFrame@235a061a{0/no_error/stop},goAwaySent=GoAwayFrame@3b0b1f4d{3/no_error/close},failure=null]}
          => org.eclipse.jetty.io.EofException: Close no_error/ (stop)
    at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onClose(HTTP2ServerConnectionFactory.java:127)
org.eclipse.jetty.io.EofException: Close no_error/ (stop)
    at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onClose(HTTP2ServerConnectionFactory.java:127) ~[http2-server-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session.notifyClose(HTTP2Session.java:1197) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session$StreamsState.terminate(HTTP2Session.java:2153) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session$StreamsState.lambda$sendGoAwayAndTerminate$16(HTTP2Session.java:2059) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.util.Callback$6.completed(Callback.java:223) ~[jetty-util-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.util.Callback$Nested.succeeded(Callback.java:396) ~[jetty-util-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.util.Callback$Nested.succeeded(Callback.java:392) ~[jetty-util-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session$ControlEntry.succeeded(HTTP2Session.java:1377) ~[http2-common-10.0.22.jar:10.0.22]
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) ~[?:?]
    at org.eclipse.jetty.http2.HTTP2Flusher.finish(HTTP2Flusher.java:323) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Flusher.succeeded(HTTP2Flusher.java:315) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:291) ~[jetty-io-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:254) ~[jetty-io-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:386) ~[jetty-io-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Flusher.process(HTTP2Flusher.java:292) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:243) ~[jetty-util-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224) ~[jetty-util-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session.frame(HTTP2Session.java:821) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session.frames(HTTP2Session.java:793) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session.control(HTTP2Session.java:775) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session$StreamsState.sendGoAway(HTTP2Session.java:2064) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session$StreamsState.sendGoAwayAndTerminate(HTTP2Session.java:2059) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session$StreamsState.lambda$onGoAway$4(HTTP2Session.java:1763) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session$StreamsState.tryRunZeroStreamsAction(HTTP2Session.java:2135) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session$StreamsState.onGoAway(HTTP2Session.java:1847) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Session.onGoAway(HTTP2Session.java:498) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Connection.onGoAway(HTTP2Connection.java:284) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.parser.BodyParser.notifyGoAway(BodyParser.java:187) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.parser.GoAwayBodyParser.onGoAway(GoAwayBodyParser.java:167) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.parser.GoAwayBodyParser.parse(GoAwayBodyParser.java:137) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.parser.Parser.parseBody(Parser.java:240) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.parser.Parser.parse(Parser.java:167) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.parser.ServerParser.parse(ServerParser.java:126) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Connection$HTTP2Producer.produce(HTTP2Connection.java:350) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455) ~[jetty-util-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248) ~[jetty-util-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:193) ~[jetty-util-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:208) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:155) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:450) ~[http2-common-10.0.22.jar:10.0.22]
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) ~[jetty-io-10.0.22.jar:10.0.22]
iamsanjay commented 1 month ago

Adding more info to it! We are benchmarking Jetty Http2 client and we are indexing document to different Shard-Replica configuration(along with different maxConnectionPerDestination and maxConcurrentStream) in Apache Solr project. However, sometimes our benchmark failed with below exception. Now It's not feasible to enable DEBUG logs in benchmark setting as it generate huge amount of logs, and also due to logging overhead we are not able to re-produce the error.

In above case, however, We tried to index one document using Jetty Http2 client and I see two different exception.

  1. Unconsumed Content
  2. Another at the end, while cleaning up the resources. close no_error

I am not sure whether Unconsumed content related to Reset cancel_stream or not. But I had to start from somewhere! 

 Server exception => org.eclipse.jetty.io.EofException: Reset cancel_stream_error
        at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onReset(HTTP2ServerConnectionFactory.java:163)
org.eclipse.jetty.io.EofException: Reset cancel_stream_error
        at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onReset(HTTP2ServerConnectionFactory.java:163) ~[http2-server-10.0.20.jar:
10.0.20]
        at org.eclipse.jetty.http2.HTTP2Stream.notifyReset(HTTP2Stream.java:853) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Stream.onReset(HTTP2Stream.java:563) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Stream.process(HTTP2Stream.java:392) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Session.onReset(HTTP2Session.java:339) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Connection.onReset(HTTP2Connection.java:260) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.BodyParser.notifyReset(BodyParser.java:139) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.ResetBodyParser.onReset(ResetBodyParser.java:94) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.ResetBodyParser.parse(ResetBodyParser.java:61) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.Parser.parseBody(Parser.java:240) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.Parser.parse(Parser.java:167) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.ServerParser.parse(ServerParser.java:126) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Connection$HTTP2Producer.produce(HTTP2Connection.java:350) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:193) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:208) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:155) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:450) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) ~[jetty-io-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) ~[jetty-io-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.Invocable.invokeNonBlocking(Invocable.java:151) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.invokeAsNonBlocking(AdaptiveExecutionStrategy.java:438) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:380) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:193) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969) [jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194) [jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149) [jetty-util-10.0.20.jar:10.0.20]
        at java.base/java.lang.Thread.run(Thread.java:829) [?:?]
sbordet commented 1 month ago

Jetty 10 is at End of Community Support

Please try Jetty 12 and report back if the problem still happens.

However, I do not understand your issue. You are looking at DEBUG logs, and we do report internal exceptions that are part of the implementation, and you should not consider these as problems.

Rather than looking at exceptions reported in DEBUG logs, please report whether you have problems at all.

From your report, looks like everything works fine, you just don't like what we report in the DEBUG logs, which should not be your concern (DEBUG logs are mostly for us, to help troubleshooting).

iamsanjay commented 1 month ago

Thanks for @sbordet for taking a look!

Apologies for mixing up the exception. But below is the exception that I am mainly concerned with which comes right after after benchmark stops abruptly. And DEBUG is not enabled here. 

Server exception => org.eclipse.jetty.io.EofException: Reset cancel_stream_error
        at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onReset(HTTP2ServerConnectionFactory.java:163)
org.eclipse.jetty.io.EofException: Reset cancel_stream_error
        at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onReset(HTTP2ServerConnectionFactory.java:163) ~[http2-server-10.0.20.jar:
10.0.20]
        at org.eclipse.jetty.http2.HTTP2Stream.notifyReset(HTTP2Stream.java:853) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Stream.onReset(HTTP2Stream.java:563) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Stream.process(HTTP2Stream.java:392) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Session.onReset(HTTP2Session.java:339) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Connection.onReset(HTTP2Connection.java:260) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.BodyParser.notifyReset(BodyParser.java:139) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.ResetBodyParser.onReset(ResetBodyParser.java:94) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.ResetBodyParser.parse(ResetBodyParser.java:61) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.Parser.parseBody(Parser.java:240) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.Parser.parse(Parser.java:167) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.parser.ServerParser.parse(ServerParser.java:126) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Connection$HTTP2Producer.produce(HTTP2Connection.java:350) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:455) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:248) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:193) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:208) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:155) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:450) ~[http2-common-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) ~[jetty-io-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) ~[jetty-io-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.Invocable.invokeNonBlocking(Invocable.java:151) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.invokeAsNonBlocking(AdaptiveExecutionStrategy.java:438) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:380) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:193) ~[jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969) [jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194) [jetty-util-10.0.20.jar:10.0.20]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149) [jetty-util-10.0.20.jar:10.0.20]
        at java.base/java.lang.Thread.run(Thread.java:829) [?:?]
iamsanjay commented 1 month ago

Yes +1 to upgrade and see If the error still persists.

iamsanjay commented 1 month ago

The main issue is that the benchmark stops unexpectedly before completion. We're currently debugging this problem, as it might be related to Apache Solr. However, I've also encountered the Reset cancel stream error exception, so I wanted to ask if this is something I should be concerned about.

joakime commented 1 month ago

@iamsanjay like @sbordet said, Jetty 10 is at end of community support.

If this occurs on Jetty 12 we can address it, otherwise this issue is considered out of support.

iamsanjay commented 1 month ago

I completely understand, @joakime. I’ll look into the upgrade. I found the documentation on upgrading, though it's specifically for moving from Jetty 11 to 12, but I can start with that. Apache Solr is still on Java 11, but we're in the process of upgrading to Java 21. Once that's done, we can focus on this task. Thank you, @joakime and @sbordet!