fasten-project / fasten

Analyse package dependency networks at the call graph level
https://www.fasten-project.eu
Apache License 2.0
91 stars 28 forks source link

CGMerger throws OutOfMemoryException #427

Open vigna opened 2 years ago

vigna commented 2 years ago

On branch develop, trying to merge with CGMerger the dependencies related to revision fasten://io.7mind.izumi:distage-extension-config_2.12$1.0.6 (id=76535) causes an out-of-memory exception, even using 128GB of RAM.

To replicate, try

echo '+fasten://io.7mind.izumi:distage-extension-config_2.12$1.0.6' | java -Xmx128G eu.fasten.core.search.SearchEngine jdbc:postgresql://monster:5432/fasten_java fastenro /mnt/fasten/data/callable-index/ foo

on monster. Note that the SearchEngine class has no correlation with the problem—it is just an easy way to run CGMerger over the revision above. After about 20 minutes, CGMerger throws the following exception:

Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
    at java.base/java.util.Arrays.copyOf(Arrays.java:3689)
    at java.base/java.util.ArrayList.grow(ArrayList.java:238)
    at java.base/java.util.ArrayList.addAll(ArrayList.java:710)
    at eu.fasten.core.merge.CGMerger.getAllChildren(CGMerger.java:757)
    at eu.fasten.core.merge.CGMerger.getAllChildren(CGMerger.java:757)
    at eu.fasten.core.merge.CGMerger.getAllChildren(CGMerger.java:757)
    at eu.fasten.core.merge.CGMerger.getAllChildren(CGMerger.java:757)
    at eu.fasten.core.merge.CGMerger.createUniversalCHA(CGMerger.java:701)
    at eu.fasten.core.merge.CGMerger.<init>(CGMerger.java:222)
    at eu.fasten.core.search.SearchEngine.from(SearchEngine.java:511)
    at eu.fasten.core.search.SearchEngine.fromRevision(SearchEngine.java:479)
    at eu.fasten.core.search.SearchEngine.fromRevision(SearchEngine.java:464)
    at eu.fasten.core.search.SearchEngine.main(SearchEngine.java:736)

Other revisions from the same group id merge without problems.

vigna commented 2 years ago

A similar behavior occurs with id=104122.

proksch commented 2 years ago

This looks indeed like a bug. @ashkboos is on vacation right now, but will look into this as soon as he is back.

vigna commented 2 years ago

Also with 1122038, 132690, 132691, and 573488.