eclipse-ee4j / grizzly

Grizzly
https://eclipse-ee4j.github.io/grizzly
Other
147 stars 69 forks source link

Grizzly HTTP2 Session/Stream - Memory Leak #2125

Closed bvsvas closed 1 year ago

bvsvas commented 3 years ago

Problem Description:

Grizzly HTTP2 Session/Stream memory is being increased in the first 10mins till the max memory and GC was also stopped reclaiming memory. The server is not able to recover from this situation. The same issue was not reproducible when I switch-off HTTP2 support and it works well.

Environment:

JDK Version: 1.8.0_u271 with G1 GC Operating System: Windows Database: N/A

Client: OkayHttp3 Server: Grizzly Http2 with Jersey Dependencies:

I have collected Heap dump and Flight recorder data.

Here are the attached images of Memory and Top Consumers from the Heap dump and Flight recorder:

image image

image image image

Steps to Reproduce:

1) Grizzly + Jersey Server can be created using the archetype below: (Add simple Rest Control with GET/POST method, Enable HTTP2 secured) mvn archetype:generate -DarchetypeGroupId=org.glassfish.jersey.archetypes -DarchetypeArtifactId=jersey-quickstart-grizzly2 -DgroupId=com.example.srini.rest -DartifactId=jersey-web-service -Dpackage=com.example.srini.rest -DarchetypeVersion=3.0.0

2) Simple OkayHttp3 Client to invoke REST Endpoint (Hit endpoint --> 10 requests per seconds continuously)

Impact of Issue

Unable to make use of HTTP2

dmatej commented 3 years ago

Hi, I tried to reproduce it but everything works as expected:

I had to extend the archetype as it used just plain HTTP, so you have to provide complete reproducer, because we probably used different implementations.

My test executed the same request for 5 minutes, with heap limited to 200 megabytes. Server and client processed 1614761 requests with this result in visualvm:

image

The first idea when I'm looking on your screenshot is that you create new session for each request, but you don't process those requests or responses completely, then HTTP/2 maybe waits for the client, but I don't know. I will create PR to grizzly with tests I created.

bvsvas commented 3 years ago

Thanks David for looking into it. Could you please share the PR link? I will upload reproducible Grizzly Server and Client in the GitHub soon, You can take a look at it.  

dmatej commented 3 years ago

I created PR with the test for Jersey here: https://github.com/eclipse-ee4j/jersey/pull/4740