WALA runs out of memory on Tomcat

[reprogrammer]

WALA runs out of memory and throws the following exception when Keshmesh is run on Tomcat. This exception is thrown while the Findbugs progress bar says that it's analyzing org.apache.tomcat.jni.FileInfo.

java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2882)
    at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
    at java.lang.StringBuilder.append(StringBuilder.java:119)
    at java.lang.StringBuilder.append(StringBuilder.java:115)
    at java.lang.Throwable.printStackTrace(Throwable.java:512)
    at org.eclipse.jdt.internal.compiler.util.Util.getExceptionSummary(Util.java:452)
    at org.eclipse.jdt.internal.compiler.Compiler.handleInternalException(Compiler.java:583)
    at org.eclipse.jdt.core.dom.CompilationUnitResolver.handleInternalException(CompilationUnitResolver.java:350)
    at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:933)
    at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:577)
    at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:888)
    at com.ibm.wala.cast.java.translator.jdt.JDTSourceModuleTranslator.loadAllSources(JDTSourceModuleTranslator.java:140)
    at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.loadAllSources(JavaSourceLoaderImpl.java:428)
    at com.ibm.wala.classLoader.ClassLoaderImpl.init(ClassLoaderImpl.java:478)
    at com.ibm.wala.cast.java.translator.jdt.JDTClassLoaderFactory.makeNewClassLoader(JDTClassLoaderFactory.java:67)
    at com.ibm.wala.classLoader.ClassLoaderFactoryImpl.getLoader(ClassLoaderFactoryImpl.java:68)
    at com.ibm.wala.ipa.cha.ClassHierarchy.(ClassHierarchy.java:229)
    at com.ibm.wala.ipa.cha.ClassHierarchy.(ClassHierarchy.java:175)
    at com.ibm.wala.ipa.cha.ClassHierarchy.make(ClassHierarchy.java:1172)
    at com.ibm.wala.cast.java.client.JavaSourceAnalysisEngine.buildClassHierarchy(JavaSourceAnalysisEngine.java:130)
    at com.ibm.wala.client.AbstractAnalysisEngine.defaultCallGraphBuilder(AbstractAnalysisEngine.java:292)
    at com.ibm.wala.client.AbstractAnalysisEngine.buildDefaultCallGraph(AbstractAnalysisEngine.java:301)
    at edu.illinois.keshmesh.detector.Main.initAnalysis(Main.java:53)
    at edu.illinois.keshmesh.detector.Main.initAndPerformAnalysis(Main.java:40)
    at edu.illinois.keshmesh.ui.findbugs.KeshmeshFindBugsDetector.visitClassContext(KeshmeshFindBugsDetector.java:80)
    at edu.umd.cs.findbugs.DetectorToDetector2Adapter.visitClass(DetectorToDetector2Adapter.java:74)
    at edu.umd.cs.findbugs.FindBugs2.analyzeApplication(FindBugs2.java:1109)
    at edu.umd.cs.findbugs.FindBugs2.execute(FindBugs2.java:266)
    at de.tobject.findbugs.builder.FindBugsWorker.runFindBugs(FindBugsWorker.java:288)
    at de.tobject.findbugs.builder.FindBugsWorker.work(FindBugsWorker.java:198)
    at de.tobject.findbugs.actions.FindBugsAction$StartedFromViewJob.runWithProgress(FindBugsAction.java:265)

[reprogrammer]

WALA seems to run out of memory in com.ibm.wala.cast.java.translator.jdt.JDTSourceModuleTranslator.loadAllSources(Set).

We increased the memory by using the following command line arguments.

-XX:MaxPermSize=512m -Xms40m -Xmx2048m

This lets WALA run more. But, this time it raises the following exception in com.ibm.wala.cast.java.translator.jdt.JDTSourceModuleTranslator.loadAllSources(Set).

edu.illinois.keshmesh.detector.exception.Exceptions$WALAInitializationException: java.lang.ClassCastException: com.ibm.wala.classLoader.ShrikeClass cannot be cast to com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl$JavaClass
    at edu.illinois.keshmesh.detector.Main.initAnalysis(Main.java:60)
    at edu.illinois.keshmesh.detector.Main.initAndPerformAnalysis(Main.java:40)
    at edu.illinois.keshmesh.ui.findbugs.KeshmeshFindBugsDetector.visitClassContext(KeshmeshFindBugsDetector.java:80)
    at edu.umd.cs.findbugs.DetectorToDetector2Adapter.visitClass(DetectorToDetector2Adapter.java:74)
    at edu.umd.cs.findbugs.FindBugs2.analyzeApplication(FindBugs2.java:1109)
    at edu.umd.cs.findbugs.FindBugs2.execute(FindBugs2.java:266)
    at de.tobject.findbugs.builder.FindBugsWorker.runFindBugs(FindBugsWorker.java:288)
    at de.tobject.findbugs.builder.FindBugsWorker.work(FindBugsWorker.java:198)
    at de.tobject.findbugs.actions.FindBugsAction$StartedFromViewJob.runWithProgress(FindBugsAction.java:265)
    at de.tobject.findbugs.FindBugsJob.run(FindBugsJob.java:61)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: java.lang.ClassCastException: com.ibm.wala.classLoader.ShrikeClass cannot be cast to com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl$JavaClass
    at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.defineField(JavaSourceLoaderImpl.java:444)
    at com.ibm.wala.cast.java.translator.JavaCAst2IRTranslator.defineField(JavaCAst2IRTranslator.java:229)
    at com.ibm.wala.cast.ir.translator.AstTranslator.leaveFieldEntity(AstTranslator.java:2371)
    at com.ibm.wala.cast.tree.visit.CAstVisitor.visitEntities(CAstVisitor.java:145)
    at com.ibm.wala.cast.tree.visit.CAstVisitor.visitScopedEntities(CAstVisitor.java:122)
    at com.ibm.wala.cast.tree.visit.CAstVisitor.visitScopedEntities(CAstVisitor.java:114)
    at com.ibm.wala.cast.tree.visit.CAstVisitor.visitEntities(CAstVisitor.java:158)
    at com.ibm.wala.cast.tree.visit.CAstVisitor.visitScopedEntities(CAstVisitor.java:122)
    at com.ibm.wala.cast.tree.visit.CAstVisitor.visitScopedEntities(CAstVisitor.java:114)
    at com.ibm.wala.cast.tree.visit.CAstVisitor.visitEntities(CAstVisitor.java:158)
    at com.ibm.wala.cast.tree.visit.CAstVisitor.visitScopedEntities(CAstVisitor.java:122)
    at com.ibm.wala.cast.tree.visit.CAstVisitor.visitScopedEntities(CAstVisitor.java:114)
    at com.ibm.wala.cast.tree.visit.CAstVisitor.visitEntities(CAstVisitor.java:138)
    at com.ibm.wala.cast.ir.translator.AstTranslator.walkEntities(AstTranslator.java:3611)
    at com.ibm.wala.cast.ir.translator.AstTranslator.translate(AstTranslator.java:3682)
    at com.ibm.wala.cast.java.translator.JavaCAst2IRTranslator.translate(JavaCAst2IRTranslator.java:51)
    at com.ibm.wala.cast.java.translator.Java2IRTranslator.translate(Java2IRTranslator.java:76)
    at com.ibm.wala.cast.java.translator.jdt.JDTSourceModuleTranslator$1.acceptAST(JDTSourceModuleTranslator.java:148)
    at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:883)
    at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:577)
    at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:888)
    at com.ibm.wala.cast.java.translator.jdt.JDTSourceModuleTranslator.loadAllSources(JDTSourceModuleTranslator.java:140)
    at com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl.loadAllSources(JavaSourceLoaderImpl.java:428)
    at com.ibm.wala.classLoader.ClassLoaderImpl.init(ClassLoaderImpl.java:478)
    at com.ibm.wala.cast.java.translator.jdt.JDTClassLoaderFactory.makeNewClassLoader(JDTClassLoaderFactory.java:67)
    at com.ibm.wala.classLoader.ClassLoaderFactoryImpl.getLoader(ClassLoaderFactoryImpl.java:68)
    at com.ibm.wala.ipa.cha.ClassHierarchy.(ClassHierarchy.java:229)
    at com.ibm.wala.ipa.cha.ClassHierarchy.(ClassHierarchy.java:175)
    at com.ibm.wala.ipa.cha.ClassHierarchy.make(ClassHierarchy.java:1172)
    at com.ibm.wala.cast.java.client.JavaSourceAnalysisEngine.buildClassHierarchy(JavaSourceAnalysisEngine.java:130)
    at com.ibm.wala.client.AbstractAnalysisEngine.defaultCallGraphBuilder(AbstractAnalysisEngine.java:292)
    at com.ibm.wala.client.AbstractAnalysisEngine.buildDefaultCallGraph(AbstractAnalysisEngine.java:301)
    at edu.illinois.keshmesh.detector.Main.initAnalysis(Main.java:53)
    ... 10 more

We think this exception is related to issue #6 because it fails when trying to define a field of an enum in a declaration of an annotation type. Specifically, the exception is thrown when WALA is trying to add the field CONTAINER to <Primordial,Ljavax/annotation/Resource$AuthenticationType> because Authentication type is a Primordial class and corresponds to com.ibm.wala.classLoader.ShrikeClass. But, com.ibm.wala.classLoader.ShrikeClass cannot be cast to com.ibm.wala.cast.java.loader.JavaSourceLoaderImpl$JavaClass.

[reprogrammer]

The problem is not specific to annotations. The problem is that if we redefine a JDK class in the source code, WALA uses the JDK version of that class. This confuses WALA during the JavaCAst to IR translation.

[reprogrammer]

We fixed the order of loading classes in the following patch for com.ibm.wala.core.

Index: src/com/ibm/wala/classLoader/ClassLoaderImpl.java
===================================================================
--- src/com/ibm/wala/classLoader/ClassLoaderImpl.java   (revision 4020)
+++ src/com/ibm/wala/classLoader/ClassLoaderImpl.java   (working copy)
@@ -554,16 +554,21 @@
       return arrayClassLoader.lookupClass(className, this, cha);
     }

-    // try delegating first.
+    // First, try our own namespace.
+    IClass result = loadedClasses.get(className);
+
+    if (result != null) {
+      return result;
+    }
+
+    // Second, try delegation.
     IClassLoader parent = getParent();
     if (parent != null) {
-      IClass result = parent.lookupClass(className);
+      result = parent.lookupClass(className);
       if (result != null) {
     return result;
       }
     }
-    // delegating failed. Try our own namespace.
-    IClass result = loadedClasses.get(className);
     return result;
   }

[reprogrammer]

The problem with the above patch is that it doesn't obey the Java's order of loading classes. See Understanding Extension Class Loading for further details. I think the right fix would be not to load the redefined classes of JDK using WALA's Java source code loader.

[reprogrammer]

We've switched from WALA's source front end (CAst) to the byte code front end. So, we no longer need to fix this issue.