Open jensmatw opened 2 months ago
spencergibb
commented
2 days ago By default, we use the jdk http client
This seems relevant https://www.baeldung.com/java-httpclient-connection-management#enhanced-httpclient
jdk.httpclient.maxstreams – set this property to control the maximum number of H2 streams that are permitted per HTTP connection (defaults to 100).
jensmatw
commented
2 days ago The http client is working more or less correctly: it uses the max value that the server is telling him (if it is a tomcat, it is 100 by default). The problem is, when 100 is reached, the http client just throws an exception, it has no own logic to create more connections. So currently, the gateway-mvc can't handle more than 100 (or whatever server is telling) parallel streams which is very limiting. The reactive gateway on the other hand uses webclient which brings it's own connection pool managed in a reactive thread pool or something. So it just creates more connections when streams max is reached.
I'm not sure if it is on springs resttemplate code or if there should be a "connection pool manager" in the gateway. But i think it is very bad that the 101st connection is just rejected by an exception because the gateway will not scale.
spencergibb
commented
2 days ago The reactive gateway uses netty HttpClient, not WebClient. Did you try setting the property I mentioned above?
jensmatw
commented
2 days ago The setting you mention would increase the amount of streams from client side. If the server (eg. the microservice downstream) is only accepting 100 streams (like a default tomcat) per connection, it will not work.
There was a JDK bug where the client did not respect the servers settings that are transmitted in the handshakelike part. But now it does.
As long as you are under control of the servers in your route configuration you could increase the max streams on all those webservers. But it would perform poorly if you put too many streams in a single connection.
If you have a route configured in the gateway to a tomcat. As soon as there are 101 clients sending requests to the gateway for that particular route via individual connections, the gateway dispatches all of them through a single connection to a that tomcat and the exception ("too many concurrent streams") is thrown.
spencergibb
commented
2 days ago I only control the client side in mvc gateway. It could be tomcat downstream or python or .net.
I don't understand what you think I could do with the jdk http client under my control.
Describe the bug If the downstream server accepts http2 and for example offers 100 concurrent streams (tomcat default) the http client in gateway mvc is using up to those 100 streams in one connection, then on the 101st parallel stream, it throws an exception (too many concurrent streams) even when the tomcat would allow a lot more connections (default is 200 tomcat threads). I'm not sure if this is a problem in either JDK, springs RestTemplate or SCG mvc. Somewhere should be a connection pool which uses multiple real connections instead of trying to put everything in a single one.
Sample Use a route to a default config tomcat (spring boot) with a long enough response time and add up to 101 requests in parallel (or reduce tomcat concurrent streams to a smaller number for easier testing).