search

AppThreat / atom

Atom is a novel intermediate representation for applications and a standalone tool that is powered by chen.
https://appthreat.com
Apache License 2.0
35 stars 2 forks source link

java.lang.OutOfMemoryError in javaparser #144

Open Hritik14 opened 1 month ago

Hritik14 commented 1 month ago

I'm trying out atom on a huge codebase and atom is erroring out with out of heap space.

; find . -name "*.java" |  wc -l
  123962

I've tried to run with increased memory using JAVA_OPTS but it doesn't seem to help

; JAVA_OPTS=-Xmx10240m atom usages --extract-endpoints -o app.atom --slice-outfile usages.json -l java .
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at com.github.javaparser.AbstractCharStream.getImage(AbstractCharStream.java:444)
        at com.github.javaparser.GeneratedJavaParserTokenManager.jjFillToken(GeneratedJavaParserTokenManager.java:2715)
        at com.github.javaparser.GeneratedJavaParserTokenManager.getNextToken(GeneratedJavaParserTokenManager.java:3069)
        at com.github.javaparser.GeneratedJavaParser.jj_ntk_f(GeneratedJavaParser.java:13944)
        at com.github.javaparser.GeneratedJavaParser.UnaryExpression(GeneratedJavaParser.java:4335)
        at com.github.javaparser.GeneratedJavaParser.MultiplicativeExpression(GeneratedJavaParser.java:4270)
        at com.github.javaparser.GeneratedJavaParser.AdditiveExpression(GeneratedJavaParser.java:4222)
        at com.github.javaparser.GeneratedJavaParser.ShiftExpression(GeneratedJavaParser.java:4177)
        at com.github.javaparser.GeneratedJavaParser.RelationalExpression(GeneratedJavaParser.java:4117)
        at com.github.javaparser.GeneratedJavaParser.InstanceOfExpression(GeneratedJavaParser.java:4034)
        at com.github.javaparser.GeneratedJavaParser.EqualityExpression(GeneratedJavaParser.java:3964)
        at com.github.javaparser.GeneratedJavaParser.AndExpression(GeneratedJavaParser.java:3919)
        at com.github.javaparser.GeneratedJavaParser.ExclusiveOrExpression(GeneratedJavaParser.java:3879)
        at com.github.javaparser.GeneratedJavaParser.InclusiveOrExpression(GeneratedJavaParser.java:3839)
        at com.github.javaparser.GeneratedJavaParser.ConditionalAndExpression(GeneratedJavaParser.java:3799)
        at com.github.javaparser.GeneratedJavaParser.ConditionalOrExpression(GeneratedJavaParser.java:3754)
        at com.github.javaparser.GeneratedJavaParser.ConditionalExpression(GeneratedJavaParser.java:3722)
        at com.github.javaparser.GeneratedJavaParser.Expression(GeneratedJavaParser.java:3533)
        at com.github.javaparser.GeneratedJavaParser.ArgumentList(GeneratedJavaParser.java:5339)
        at com.github.javaparser.GeneratedJavaParser.Arguments(GeneratedJavaParser.java:5305)
        at com.github.javaparser.GeneratedJavaParser.PrimarySuffixWithoutSuper(GeneratedJavaParser.java:5099)
        at com.github.javaparser.GeneratedJavaParser.PrimarySuffix(GeneratedJavaParser.java:5044)
        at com.github.javaparser.GeneratedJavaParser.PrimaryExpression(GeneratedJavaParser.java:4713)
        at com.github.javaparser.GeneratedJavaParser.StatementExpression(GeneratedJavaParser.java:6118)
        at com.github.javaparser.GeneratedJavaParser.Statement(GeneratedJavaParser.java:5679)
        at com.github.javaparser.GeneratedJavaParser.BlockStatement(GeneratedJavaParser.java:5933)
        at com.github.javaparser.GeneratedJavaParser.Statements(GeneratedJavaParser.java:2795)
        at com.github.javaparser.GeneratedJavaParser.Block(GeneratedJavaParser.java:5810)
        at com.github.javaparser.GeneratedJavaParser.MethodDeclaration(GeneratedJavaParser.java:2188)
        at com.github.javaparser.GeneratedJavaParser.ClassOrInterfaceBodyDeclaration(GeneratedJavaParser.java:1785)
        at com.github.javaparser.GeneratedJavaParser.ClassOrInterfaceBody(GeneratedJavaParser.java:1281)
        at com.github.javaparser.GeneratedJavaParser.ClassOrInterfaceDeclaration(GeneratedJavaParser.java:538)

I think this might be related: https://github.com/javaparser/javaparser/issues/1391 I'm not experienced in scala, and I suppose these details are not sufficient to pinpoint the issue. Is there any more info I can provide ?

prabhu commented 1 month ago

@Hritik14, we may need over 32GB RAM for that many files. Can you execute multiple times from within specific modules?