search

payara / Payara

Payara Server is an open source middleware platform that supports reliable and secure deployments of Java EE (Jakarta EE) and MicroProfile applications in any environment: on premise, in the cloud or hybrid.
http://www.payara.fish
Other
879 stars 301 forks source link

Bug Report: Memory leak in Payara >= 6.2023.2 with file upload / FISH-7667 #6324

Open georgwolf opened 1 year ago

georgwolf commented 1 year ago

Brief Summary

Using a Faces file upload component in a form slowly increases memory usage and will eventually lead to running out of heap space.

Expected Outcome

No memory leak occuring

Current Outcome

When having a Faces h:inputFile component on a page and then continuously submitting the form, the number of org.apache.catalina.fileupload.PartItem and related objects are constantly growing. They aren't garbage collected even when triggering an explicit GC run. This leads to a growing amount of memory being consumed and will ultimately lead to an out of memory situation. grafik

The objects are referenced by jdk.internal.ref.CleanerImpl$PhantomCleanableRef -> jdk.internal.ref.CleanerImpl when checking the heap dump: grafik

The memory leak appears first in Payara 6.2023.2 and when comparing the heap dump against an unaffected Payara 6.2023.1 a notable difference is that there's now a second GC root of type jdk.internal.ref.CleanerImpl which is holding all the references to PhantomCleanableRef -> PartItem grafik as compared to heap dump from 6.2023.1: grafik

I suspect the cause of this is FISH-6432 (Commits https://github.com/payara/Payara/commit/216695146b1a0722028b78fbb44209f0ec330a6b and https://github.com/payara/Payara/commit/61bcdfaeade0027af3e078801573e3c49ebdac23)

Reproducer

A demo of this issue using Payara Micro is available as a Maven project here: https://github.com/georgwolf/payara6_memoryleak

Run it with mvn package payara-micro:start and then go to http://localhost:8080/memleak/index.xhtml, submit the form using the "Upload" button and watch the number of retained PartItem etc. grow with every submit. Running a GC using the GC button on the page doesn't decrease the number of PartItem etc. objects. I've also included a small Selenium test (FileUploadControllerTest) to do this automatically. The demo includes JavaMelody for easier monitoring of the issue (available at http://localhost:8080/memleak/monitoring)

To clearly see the issue, compare the behaviour to Payara 6.2023.1 which does not have this memory issue using the Maven profile 6_2023_1: mvn package payara-micro:start -P 6_2023_1 Even when submitting the form multiple times, no PartItem are accruing there.

Operating System

Windows 10 Pro

JDK Version

Tested on Oracle JDK 11.0.12 and Zulu 17.0.6

Payara Distribution

Payara Micro, Payara Docker Image

kalinchan commented 1 year ago

Hi @georgwolf,

Thank you for your detailed bug report and reproducer, I have been able to reproduce this issue and have escalated the issue to engineering under FISH-7667

eric-ionos commented 4 months ago

is there any action on this topic?