search

eclipse-mat / mat

The Eclipse Memory Analyzer is a fast and feature-rich Java heap dump analyzer that helps you find memory leaks and reduce memory consumption.
https://eclipse.dev/mat/
Eclipse Public License 2.0
26 stars 8 forks source link

Slow parsing due to huge ClassImpl.subClasses in GarbageCleaner #59

Open turbanoff opened 3 weeks ago

turbanoff commented 3 weeks ago

I have a heap dump, which Eclipse MAT have troubles to open. image

Thread dump always show this stack trace:

"Worker-1: Parsing heap dump from 'C:\Users\turbanov\Documents\broqa_dumps\java_pid789.hprof\java_pid789.hprof'@8474" prio=5 tid=0x29 nid=NA runnable
  java.lang.Thread.State: RUNNABLE
      at org.eclipse.mat.parser.model.ClassImpl.removeSubClass(ClassImpl.java:467)
      at org.eclipse.mat.parser.internal.GarbageCleaner.clean(GarbageCleaner.java:238)
      at org.eclipse.mat.parser.internal.SnapshotFactoryImpl.parse(SnapshotFactoryImpl.java:466)
      at org.eclipse.mat.parser.internal.SnapshotFactoryImpl.openSnapshot(SnapshotFactoryImpl.java:224)
      at org.eclipse.mat.snapshot.SnapshotFactory.openSnapshot(SnapshotFactory.java:149)
      at org.eclipse.mat.ui.snapshot.ParseHeapDumpJob.run(ParseHeapDumpJob.java:99)
      at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

I was able to attach debugger. Seems issue with huge lists, used by org.eclipse.mat.parser.internal.GarbageCleaner First one is - org.eclipse.mat.parser.model.ClassImpl#subClasses image Second one is - classes2remove image

I think at least org.eclipse.mat.parser.model.ClassImpl#subClasses could be converted to HashSet/LinkedHashSet to make ClassImpl.removeSubClass to be run in O(1).

turbanoff commented 2 weeks ago

Yes, solution to use LinkedHashSet instead of ArrayList for ClassImpl.subClasses works. After changing I was able to open the heap dump in reasonable time.

jasonk000 commented 2 weeks ago

looks related in #57

jasonk000 commented 2 weeks ago

@turbanoff , thank you for the issue/analysis/fix. Are you able to share the heapdump or do any more analysis here?

As we saw in #57, the patch to GarbageCleaner to use FJ pool seems to have triggered this. https://github.com/eclipse-mat/mat/commit/7c2cb094fc6119999e14e1d8a04fe15c37c31f34

But the patch does not change the ClassImpl, which makes me wonder -- does the new ObjectMarker implementation have a different bug that is leading to a larger list than expected?

turbanoff commented 2 weeks ago

Here the dump, which Eclipse MAT opens very slowly. Was able to create it with quite small program. java_pid12932.hprof.gz