search

jetty / jetty.project

Eclipse Jetty® - Web Container & Clients - supports HTTP/2, HTTP/1.1, HTTP/1.0, websocket, servlets, and more
https://eclipse.dev/jetty
Other
3.85k stars 1.91k forks source link

Header size limitation in http2 client #12057

Closed marcstern closed 1 month ago

marcstern commented 2 months ago

Jetty version(s) 12.0.11

Jetty Environment Jmeter (client lib)

Java version/vendor (use: java -version) openjdk version "21.0.2" 2024-01-16 LTS OpenJDK 64-Bit Server VM Microsoft-8905927 (build 21.0.2+13-LTS, mixed mode, sharing)

OS type/version Windows 11

Description I send, with Jmeter, in HTTP/2 + HTTPS 4 headers of 8 K each and I get the following exception

org.eclipse.jetty.http2.hpack.HpackException$SessionException: Could not hpack encode GET{u=https://vh1.waf-test.approach.be:443/TestSecHeadersSizeMax100k,HTTP/2.0,h=15,cl=-1,p=null}
    at org.eclipse.jetty.http2.hpack.HpackEncoder.encode(HpackEncoder.java:278)
    at org.eclipse.jetty.http2.generator.FrameGenerator.encode(FrameGenerator.java:56)
    at org.eclipse.jetty.http2.generator.HeadersGenerator.generateHeaders(HeadersGenerator.java:65)
    at org.eclipse.jetty.http2.generator.HeadersGenerator.generate(HeadersGenerator.java:52)
    at org.eclipse.jetty.http2.generator.Generator.control(Generator.java:86)
    at org.eclipse.jetty.http2.HTTP2Session$ControlEntry.generate(HTTP2Session.java:1245)
    at org.eclipse.jetty.http2.HTTP2Flusher.process(HTTP2Flusher.java:215)
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:232)
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:214)
    at org.eclipse.jetty.http2.HTTP2Session.frame(HTTP2Session.java:777)
    at org.eclipse.jetty.http2.HTTP2Session.frames(HTTP2Session.java:749)
    at org.eclipse.jetty.http2.client.HTTP2ClientConnectionFactory$HTTP2ClientConnection.onOpen(HTTP2ClientConnectionFactory.java:116)
    at org.eclipse.jetty.io.AbstractEndPoint.upgrade(AbstractEndPoint.java:451)
    at org.eclipse.jetty.io.NegotiatingClientConnection.replaceConnection(NegotiatingClientConnection.java:117)
    at org.eclipse.jetty.io.NegotiatingClientConnection.onFillable(NegotiatingClientConnection.java:87)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:319)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
    at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:558)
    at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:379)
    at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:146)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
    at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:412)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:381)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:268)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:190)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:894)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1038)
    ... 1 more
Caused by: java.nio.BufferOverflowException
    at java.base/java.nio.Buffer.nextPutIndex(Buffer.java:725)
    at java.base/java.nio.DirectByteBuffer.put(DirectByteBuffer.java:355)
    at org.eclipse.jetty.http2.hpack.Huffman.encode(Huffman.java:504)
    at org.eclipse.jetty.http2.hpack.Huffman.encode(Huffman.java:434)
    at org.eclipse.jetty.http2.hpack.HpackEncoder.encodeValue(HpackEncoder.java:462)
    at org.eclipse.jetty.http2.hpack.HpackEncoder.encode(HpackEncoder.java:363)
    at org.eclipse.jetty.http2.hpack.HpackEncoder.encode(HpackEncoder.java:249)
    ... 28 more

The same request in HTTP (h2c):

 java.io.IOException: frame_size_error/invalid_frame_length
    at org.eclipse.jetty.http2.HTTP2Session.toFailure(HTTP2Session.java:591)
    at org.eclipse.jetty.http2.HTTP2Session$StreamsState.onSessionFailure(HTTP2Session.java:1948)
    at org.eclipse.jetty.http2.HTTP2Session.onSessionFailure(HTTP2Session.java:536)
    at org.eclipse.jetty.http2.HTTP2Session.onConnectionFailure(HTTP2Session.java:531)
    at org.eclipse.jetty.http2.parser.Parser$Listener$Wrapper.onConnectionFailure(Parser.java:409)
    at org.eclipse.jetty.http2.HTTP2Connection$ParserListener.onConnectionFailure(HTTP2Connection.java:414)
    at org.eclipse.jetty.http2.parser.BodyParser.notifyConnectionFailure(BodyParser.java:218)
    at org.eclipse.jetty.http2.parser.BodyParser.connectionFailure(BodyParser.java:210)
    at org.eclipse.jetty.http2.parser.Parser.connectionFailure(Parser.java:204)
    at org.eclipse.jetty.http2.parser.Parser.parseHeader(Parser.java:150)
    at org.eclipse.jetty.http2.parser.Parser.parse(Parser.java:116)
    at org.eclipse.jetty.http2.HTTP2Connection$HTTP2Producer.produce(HTTP2Connection.java:278)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:446)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:239)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:190)
    at org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:208)
    at org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:155)
    at org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:378)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
    at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
    at org.eclipse.jetty.util.thread.Invocable.invokeNonBlocking(Invocable.java:151)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.invokeAsNonBlocking(AdaptiveExecutionStrategy.java:429)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:371)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:268)
    at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:190)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:894)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1038)
    ... 1 more

I filled an issue fro the Jmeter plugin, but they say thay cannot control anything in the jetty lib: https://github.com/Blazemeter/jmeter-http2-plugin/issues/42

Could there be a property to manage the allowed size? Thanks a lot

PS: The stack trace is from version 11, but I have the same problem with version 12

sbordet commented 1 month ago

Is there a reason to send 4 such headers so large in a request?

This is typically an attack, and servers protect from such attacks by limiting the size of the headers.

In any case, you want to configure the server with HttpConfiguration.setRequestHeaderSize() with a value larger than your headers, so say about 40 KiB (or 90 KiB, but really?).

The server will send this setting to the client, and the client will be allowed to send such large requests.

marcstern commented 1 month ago

Yes, we have a very good reason to send these (proprietary) headers and the server accepts them. The problem is that we cannot test it with JMeter because the client lib refuses to send them.

The server will send this setting to the client

Do you mean that the library will allow larger headers it it previously receives some response header? Which one?

sbordet commented 1 month ago

Do you mean that the library will allow larger headers it it previously receives some response header? Which one?

No, this is negotiated at the HTTP/2 level with a SETTINGS frame.

The configuration from HttpConfiguration.setRequestHeaderSize() is copied into a SETTINGS frame at connection establishment, sent to the client, which receives it and configures the HPACK encoder to allow headers as large as the server is willing to accept.