search

pascal-lab / Tai-e

An easy-to-learn/use static analysis framework for Java
https://tai-e.pascal-lab.net/docs/index.html
GNU Lesser General Public License v3.0
1.45k stars 175 forks source link

Reference analysis bug for `U.class` in Soot frontend #18

Closed FXTi closed 1 year ago

FXTi commented 1 year ago

Describe the bug

I tried to do pointer analysis on MapReduce, and get exception:

Exception in thread "main" java.lang.RuntimeException: This operation requires resolving level HIERARCHY but org.apache.hadoop.hbase.mapreduce.RowCounter is at resolving level DANGLING
If you are extending Soot, try to add the following call before calling soot.Main.main(..):
Scene.v().addBasicClass(org.apache.hadoop.hbase.mapreduce.RowCounter,HIERARCHY);
Otherwise, try whole-program mode (-w).
    at soot.SootClass.checkLevelIgnoreResolving(SootClass.java:209)
    at soot.SootClass.checkLevel(SootClass.java:191)
    at soot.SootClass.getSuperclass(SootClass.java:855)
    at pascal.taie.frontend.soot.SootClassBuilder.getSuperClass(SootClassBuilder.java:76)
    at pascal.taie.language.classes.JClass.build(JClass.java:106)
    at pascal.taie.frontend.soot.SootClassBuilder.build(SootClassBuilder.java:53)
    at pascal.taie.frontend.soot.SootClassLoader.loadClass(SootClassLoader.java:67)
    at pascal.taie.language.type.ClassType.getJClass(ClassType.java:49)
    at pascal.taie.analysis.pta.plugin.util.CSObjs.toClass(CSObjs.java:70)
    at pascal.taie.analysis.pta.plugin.reflection.StringBasedModel.lambda$getMethod$6(StringBasedModel.java:109)
    at java.base/java.lang.Iterable.forEach(Iterable.java:75)
    at pascal.taie.analysis.pta.plugin.reflection.StringBasedModel.getMethod(StringBasedModel.java:108)
    at pascal.taie.analysis.pta.plugin.util.AbstractModel.lambda$handleNewPointsToSet$0(AbstractModel.java:128)
    at java.base/java.lang.Iterable.forEach(Iterable.java:75)
    at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1092)
    at pascal.taie.analysis.pta.plugin.util.AbstractModel.handleNewPointsToSet(AbstractModel.java:124)
    at pascal.taie.analysis.pta.plugin.reflection.ReflectionAnalysis.onNewPointsToSet(ReflectionAnalysis.java:92)
    at pascal.taie.analysis.pta.plugin.CompositePlugin.lambda$onNewPointsToSet$1(CompositePlugin.java:105)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at pascal.taie.analysis.pta.plugin.CompositePlugin.onNewPointsToSet(CompositePlugin.java:105)
    at pascal.taie.analysis.pta.core.solver.DefaultSolver.analyze(DefaultSolver.java:255)
    at pascal.taie.analysis.pta.core.solver.DefaultSolver.solve(DefaultSolver.java:222)
    at pascal.taie.analysis.pta.PointerAnalysis.runAnalysis(PointerAnalysis.java:118)
    at pascal.taie.analysis.pta.PointerAnalysis.analyze(PointerAnalysis.java:106)
    at pascal.taie.analysis.pta.PointerAnalysis.analyze(PointerAnalysis.java:63)
    at pascal.taie.analysis.AnalysisManager.runProgramAnalysis(AnalysisManager.java:93)
    at pascal.taie.analysis.AnalysisManager.runAnalysis(AnalysisManager.java:71)
    at pascal.taie.analysis.AnalysisManager.lambda$execute$0(AnalysisManager.java:60)
    at pascal.taie.util.Timer.lambda$runAndCount$0(Timer.java:112)
    at pascal.taie.util.Timer.runAndCount(Timer.java:93)
    at pascal.taie.util.Timer.runAndCount(Timer.java:111)
    at pascal.taie.util.Timer.runAndCount(Timer.java:107)
    at pascal.taie.analysis.AnalysisManager.lambda$execute$1(AnalysisManager.java:59)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at pascal.taie.analysis.AnalysisManager.execute(AnalysisManager.java:59)
    at pascal.taie.Main.executePlan(Main.java:142)
    at pascal.taie.Main.lambda$main$0(Main.java:56)
    at pascal.taie.util.Timer.lambda$runAndCount$0(Timer.java:112)
    at pascal.taie.util.Timer.runAndCount(Timer.java:93)
    at pascal.taie.util.Timer.runAndCount(Timer.java:111)
    at pascal.taie.util.Timer.runAndCount(Timer.java:107)
    at pascal.taie.Main.main(Main.java:48)

Tai-e arguments

        String files = Stream.of(new File("target").listFiles())
                .filter(file -> !file.isDirectory())
                .map(File::toString)
                .collect(Collectors.joining(":"));

        Main.main(new String[]{ "-cp", files,
                "-m", "org.apache.hadoop.hbase.mapreduce.Driver",
                "-java", "8",
                "-a", "pta"
        });

Compressed target folder: https://raw.githubusercontent.com/FXTi/codeql-uboot/main/target.tar.gz

Runtime environment infomation

OS: ArchLinux IDE: IDEA version of Tai-e: 0.0.3

zhangt2333 commented 1 year ago

Try to add --allow-phantom to suppress the resolving error because of frontend according to wiki:

        String files = Stream.of(new File("target").listFiles())
                .filter(file -> !file.isDirectory())
                .map(File::toString)
-                .collect(Collectors.joining(":"));
+                .collect(Collectors.joining(File.pathSeparator));

        Main.main(new String[]{ "-cp", files,
+                "--allow-phantom", // or "-ap"
                "-m", "org.apache.hadoop.hbase.mapreduce.Driver",
                "-java", "8",
                "-a", "pta"
        });
FXTi commented 1 year ago

Thanks! It solve my problem.

zhangt2333 commented 1 year ago

After I dug into this I found it was indeed a frontend-related issue and although "--allow-phantom" can suppress the error, there are some potential pitfalls.

For example, in the following case, the frontend will not execute valid reference analysis on U.class, then the class U will not be included in the World of analysis (then Soot error occurs).

public class Test {
    public static void main(String[] args) throws Exception {
        Class<U> klass = U.class; // U is another empty class in the same directory with Test
        java.lang.reflect.Method[] declaredMethods = klass.getDeclaredMethods(); // just for using U
    }
}

"--allow-phantom" can suppress the error, but notes that some classes will be empty/ignored, that is, in the above case, U and its related process will be ignored. (There are so many U.class in the main method of org.apache.hadoop.hbase.mapreduce.Driver)

If you are very concerned about U‘s handling, then you can manually add U via --input-classes according to wiki.

It is an issue about Soot frontend, we are working on it.

FXTi commented 1 year ago

After I dug into this I found it was indeed a frontend-related issue and although "--allow-phantom" can suppress the error, there are some potential pitfalls.

For example, in the following case, the frontend will not execute valid reference analysis on U.class, then the class U will not be included in the World of analysis (then Soot error occurs).

public class Test {
    public static void main(String[] args) throws Exception {
        Class<U> klass = U.class; // U is another empty class in the same directory with Test
        java.lang.reflect.Method[] declaredMethods = klass.getDeclaredMethods(); // just for using U
    }
}

"--allow-phantom" can suppress the error, but notes that some classes will be empty/ignored, that is, in the above case, U and its related process will be ignored. (There are so many U.class in the main method of org.apache.hadoop.hbase.mapreduce.Driver)

If you are very concerned about U‘s handling, then you can manually add U via --input-classes according to wiki.

It is an issue about Soot frontend, we are working on it.

I'm curious about the progress of new frontend. Any plan? Maybe you can open a https://github.com/pascal-lab/Tai-e/projects and give out an outline of new frontend. Community contribution can boost the progress. : )