wala / WALA

T.J. Watson Libraries for Analysis, with frontends for Java, Android, and JavaScript, and may common static program analyses
http://github.com/wala/WALA
Eclipse Public License 2.0
762 stars 221 forks source link

[Android] got two invalid descriptor when building callgraph #350

Closed lczxxx123 closed 6 years ago

lczxxx123 commented 6 years ago

WALA version: 1.5.0 on mvnrepository.

APK1 link (source code && release apk): https://f-droid.org/packages/com.philliphsu.clock2/

APK2 link (source code && release apk): https://f-droid.org/packages/com.phpsysinfo/

My problem is similar to issue 324 https://github.com/wala/WALA/issues/324 and issue 348 https://github.com/wala/WALA/issues/348

when i try to build callgraph,

APK1 failed with: Exception in thread "main" java.lang.IllegalArgumentException: invalid descriptor: Landroid/view/View;

APK2 failed with: Exception in thread "main" java.lang.IllegalArgumentException: invalid descriptor: V

However, with version 1.4.3, i can build the callgraphs and use them.

I will stick two StackTraces of two APKs and code to make callgraph in comment bellow.

Thanks in advance for any help

lczxxx123 commented 6 years ago

APK1:

[Start] makeCallGraph SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. got here got here: < Application, Lcom/philliphsu/clock2/MainActivity$1, (Lcom/philliphsu/clock2/MainActivity;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$1, run()V > got here got here: < Application, Lcom/philliphsu/clock2/MainActivity, ()V > got here: < Application, Lcom/philliphsu/clock2/MainActivity, access$000(Lcom/philliphsu/clock2/MainActivity;)Lcom/philliphsu/clock2/MainActivity$SectionsPagerAdapter; > got here: < Application, Lcom/philliphsu/clock2/MainActivity, access$100(Lcom/philliphsu/clock2/MainActivity;)F > got here: < Application, Lcom/philliphsu/clock2/MainActivity, access$200(Lcom/philliphsu/clock2/MainActivity;)Landroid/graphics/drawable/Drawable; > got here: < Application, Lcom/philliphsu/clock2/MainActivity, getFabPixelOffsetForXTranslation()F > got here: < Application, Lcom/philliphsu/clock2/MainActivity, handleActionScrollToStableId(Landroid/content/Intent;Z)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity, setTabIcon(IILandroid/content/res/ColorStateList;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity, isDisplayHomeUpEnabled()Z > got here: < Application, Lcom/philliphsu/clock2/MainActivity, layoutResId()I > got here: < Application, Lcom/philliphsu/clock2/MainActivity, menuResId()I > got here: < Application, Lcom/philliphsu/clock2/MainActivity, onActivityResult(IILandroid/content/Intent;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity, onCreate(Landroid/os/Bundle;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity, onNewIntent(Landroid/content/Intent;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity, onOptionsItemSelected(Landroid/view/MenuItem;)Z > got here got here: < Application, Lcom/philliphsu/clock2/MainActivity$2, (Lcom/philliphsu/clock2/MainActivity;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$2, onPageScrolled(IFI)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$2, onPageSelected(I)V > got here got here: < Application, Lcom/philliphsu/clock2/MainActivity$$ViewBinder, ()V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$$ViewBinder, bind(Lbutterknife/ButterKnife$Finder;Lcom/philliphsu/clock2/BaseActivity;Ljava/lang/Object;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$$ViewBinder, bind(Lbutterknife/ButterKnife$Finder;Lcom/philliphsu/clock2/MainActivity;Ljava/lang/Object;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$$ViewBinder, bind(Lbutterknife/ButterKnife$Finder;Ljava/lang/Object;Ljava/lang/Object;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$$ViewBinder, unbind(Lcom/philliphsu/clock2/BaseActivity;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$$ViewBinder, unbind(Lcom/philliphsu/clock2/MainActivity;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$$ViewBinder, unbind(Ljava/lang/Object;)V > got here got here: < Application, Lcom/philliphsu/clock2/MainActivity$SectionsPagerAdapter, (Landroid/support/v4/app/FragmentManager;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$SectionsPagerAdapter, destroyItem(Landroid/view/ViewGroup;ILjava/lang/Object;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$SectionsPagerAdapter, getCount()I > got here: < Application, Lcom/philliphsu/clock2/MainActivity$SectionsPagerAdapter, getFragment(I)Landroid/support/v4/app/Fragment; > got here: < Application, Lcom/philliphsu/clock2/MainActivity$SectionsPagerAdapter, getItem(I)Landroid/support/v4/app/Fragment; > got here: < Application, Lcom/philliphsu/clock2/MainActivity$SectionsPagerAdapter, instantiateItem(Landroid/view/ViewGroup;I)Ljava/lang/Object; > got here got here: < Application, Lcom/philliphsu/clock2/MainActivity$3, (Lcom/philliphsu/clock2/MainActivity;)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$3, onClick(Landroid/view/View;)V > got here got here: < Application, Lcom/philliphsu/clock2/MainActivity$4, (Lcom/philliphsu/clock2/MainActivity;ILandroid/content/Intent;Z)V > got here: < Application, Lcom/philliphsu/clock2/MainActivity$4, run()V > Exception in thread "main" java.lang.IllegalArgumentException: invalid descriptor: Landroid/view/View; at com.ibm.wala.util.strings.StringStuff.parseForReturnTypeName(StringStuff.java:153) at com.ibm.wala.types.Descriptor.findOrCreate(Descriptor.java:64) at com.ibm.wala.types.Descriptor.findOrCreateUTF8(Descriptor.java:93) at com.ibm.wala.types.MethodReference.findOrCreate(MethodReference.java:148) at com.ibm.wala.dalvik.ssa.DexSSABuilder$SymbolicPropagator$NodeVisitor.visitInvoke(DexSSABuilder.java:792) at com.ibm.wala.dalvik.dex.instructions.Invoke.visit(Invoke.java:229) at com.ibm.wala.dalvik.ssa.AbstractIntRegisterMachine$BasicRegisterFlowProvider.flow(AbstractIntRegisterMachine.java:862) at com.ibm.wala.dalvik.ssa.AbstractIntRegisterMachine$1$1.evaluate(AbstractIntRegisterMachine.java:139) at com.ibm.wala.dalvik.ssa.AbstractIntRegisterMachine$1$1.evaluate(AbstractIntRegisterMachine.java:132) at com.ibm.wala.fixpoint.UnaryStatement.evaluate(UnaryStatement.java:35) at com.ibm.wala.fixedpoint.impl.AbstractFixedPointSolver.solve(AbstractFixedPointSolver.java:152) at com.ibm.wala.dalvik.ssa.AbstractIntRegisterMachine.solve(AbstractIntRegisterMachine.java:257) at com.ibm.wala.dalvik.ssa.DexSSABuilder.build(DexSSABuilder.java:1361) at com.ibm.wala.dalvik.classLoader.DexIRFactory$1.(DexIRFactory.java:102) at com.ibm.wala.dalvik.classLoader.DexIRFactory.makeIR(DexIRFactory.java:69) at com.ibm.wala.ssa.SSACache.findOrCreateIR(SSACache.java:78) at com.ibm.wala.ipa.callgraph.AnalysisCache.getIR(AnalysisCache.java:74) at com.ibm.wala.ipa.callgraph.propagation.cfa.ContextInsensitiveSSAInterpreter.getIR(ContextInsensitiveSSAInterpreter.java:46) at com.ibm.wala.ipa.callgraph.propagation.cfa.DefaultSSAInterpreter.getIR(DefaultSSAInterpreter.java:57) at com.ibm.wala.ipa.callgraph.propagation.cfa.DelegatingSSAContextInterpreter.getIR(DelegatingSSAContextInterpreter.java:56) at com.ibm.wala.ipa.callgraph.impl.ExplicitCallGraph$ExplicitNode.getIR(ExplicitCallGraph.java:307) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.getTargetPointerKey(SSAPropagationCallGraphBuilder.java:2191) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.processCallingConstraints(SSAPropagationCallGraphBuilder.java:1640) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.processResolvedCall(SSAPropagationCallGraphBuilder.java:1593) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.access$200(SSAPropagationCallGraphBuilder.java:98) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$ConstraintVisitor.visitInvokeInternal(SSAPropagationCallGraphBuilder.java:1139) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$ConstraintVisitor.visitInvoke(SSAPropagationCallGraphBuilder.java:1108) at com.ibm.wala.ssa.SSAInvokeInstruction.visit(SSAInvokeInstruction.java:85) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.addBlockInstructionConstraints(SSAPropagationCallGraphBuilder.java:290) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.addNodeInstructionConstraints(SSAPropagationCallGraphBuilder.java:261) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.unconditionallyAddConstraintsFromNode(SSAPropagationCallGraphBuilder.java:233) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.addConstraintsFromNode(SSAPropagationCallGraphBuilder.java:199) at com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.addConstraintsFromNewNodes(PropagationCallGraphBuilder.java:335) at com.ibm.wala.ipa.callgraph.propagation.StandardSolver.solve(StandardSolver.java:58) at com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.makeCallGraph(PropagationCallGraphBuilder.java:268) at com.lc.NPEDetector.makeCallGraph(NPEDetector.java:103) at com.lc.NPEDetector.main(NPEDetector.java:61)

APK2:

[Start] makeCallGraph SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Exception in thread "main" java.lang.IllegalArgumentException: invalid descriptor: V at com.ibm.wala.util.strings.StringStuff.parseForReturnTypeName(StringStuff.java:149) at com.ibm.wala.types.Descriptor.findOrCreate(Descriptor.java:64) at com.ibm.wala.types.Descriptor.findOrCreateUTF8(Descriptor.java:93) at com.ibm.wala.types.MethodReference.findOrCreate(MethodReference.java:148) at com.ibm.wala.dalvik.ssa.DexSSABuilder$SymbolicPropagator$NodeVisitor.visitInvoke(DexSSABuilder.java:792) at com.ibm.wala.dalvik.dex.instructions.Invoke.visit(Invoke.java:229) at com.ibm.wala.dalvik.ssa.AbstractIntRegisterMachine$BasicRegisterFlowProvider.flow(AbstractIntRegisterMachine.java:862) at com.ibm.wala.dalvik.ssa.AbstractIntRegisterMachine$1$1.evaluate(AbstractIntRegisterMachine.java:139) at com.ibm.wala.dalvik.ssa.AbstractIntRegisterMachine$1$1.evaluate(AbstractIntRegisterMachine.java:132) at com.ibm.wala.fixpoint.UnaryStatement.evaluate(UnaryStatement.java:35) at com.ibm.wala.fixedpoint.impl.AbstractFixedPointSolver.solve(AbstractFixedPointSolver.java:152) at com.ibm.wala.dalvik.ssa.AbstractIntRegisterMachine.solve(AbstractIntRegisterMachine.java:257) at com.ibm.wala.dalvik.ssa.DexSSABuilder.build(DexSSABuilder.java:1361) at com.ibm.wala.dalvik.classLoader.DexIRFactory$1.(DexIRFactory.java:102) at com.ibm.wala.dalvik.classLoader.DexIRFactory.makeIR(DexIRFactory.java:69) at com.ibm.wala.ssa.SSACache.findOrCreateIR(SSACache.java:78) at com.ibm.wala.ipa.callgraph.AnalysisCache.getIR(AnalysisCache.java:74) at com.ibm.wala.ipa.callgraph.propagation.cfa.ContextInsensitiveSSAInterpreter.getIR(ContextInsensitiveSSAInterpreter.java:46) at com.ibm.wala.ipa.callgraph.propagation.cfa.DefaultSSAInterpreter.getIR(DefaultSSAInterpreter.java:57) at com.ibm.wala.ipa.callgraph.propagation.cfa.DelegatingSSAContextInterpreter.getIR(DelegatingSSAContextInterpreter.java:56) at com.ibm.wala.ipa.callgraph.impl.ExplicitCallGraph$ExplicitNode.getIR(ExplicitCallGraph.java:307) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.getTargetPointerKey(SSAPropagationCallGraphBuilder.java:2191) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.processCallingConstraints(SSAPropagationCallGraphBuilder.java:1640) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.processResolvedCall(SSAPropagationCallGraphBuilder.java:1593) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.access$200(SSAPropagationCallGraphBuilder.java:98) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$ConstraintVisitor.visitInvokeInternal(SSAPropagationCallGraphBuilder.java:1139) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$ConstraintVisitor.visitInvoke(SSAPropagationCallGraphBuilder.java:1108) at com.ibm.wala.ssa.SSAInvokeInstruction.visit(SSAInvokeInstruction.java:85) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.addBlockInstructionConstraints(SSAPropagationCallGraphBuilder.java:290) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.addNodeInstructionConstraints(SSAPropagationCallGraphBuilder.java:261) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.unconditionallyAddConstraintsFromNode(SSAPropagationCallGraphBuilder.java:233) at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.addConstraintsFromNode(SSAPropagationCallGraphBuilder.java:199) at com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.addConstraintsFromNewNodes(PropagationCallGraphBuilder.java:335) at com.ibm.wala.ipa.callgraph.propagation.StandardSolver.solve(StandardSolver.java:58) at com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.makeCallGraph(PropagationCallGraphBuilder.java:268) at com.lc.NPEDetector.makeCallGraph(NPEDetector.java:103) at com.lc.NPEDetector.main(NPEDetector.java:61)

Code:

private void makeCallGraph() throws IOException, ClassHierarchyException, IllegalArgumentException, CallGraphBuilderCancelException {
    System.out.println("[Start] makeCallGraph");
    long startTime = System.currentTimeMillis();
    FileProvider fileProvider = new FileProvider();
    File exclusionsFile = fileProvider.getFile(exclusionFilePath);

    AnalysisScope scope = AnalysisScope.createJavaAnalysisScope();
    final InputStream fStream = exclusionsFile.exists() ? new FileInputStream(exclusionsFile) 
            : FileProvider.class.getClassLoader().getResourceAsStream(exclusionsFile.getName());
    scope.setExclusions(new FileOfClasses(fStream));
    if (AndroidJarPath==null || APKPath == null) {
        System.out.println("AndroidJarPath or APKPath is null!!!!!!!");
        System.exit(1);
    }
    final JarFile AndroidJar = new JarFile(new File(AndroidJarPath));
    scope.addToScope(ClassLoaderReference.Primordial, AndroidJar);
    scope.setLoaderImpl(ClassLoaderReference.Application, "com.ibm.wala.dalvik.classLoader.WDexClassLoaderImpl");
    scope.addToScope(ClassLoaderReference.Application,DexFileModule.make(new File(APKPath)));

    final IClassHierarchy cha = ClassHierarchyFactory.make(scope);
    IAnalysisCacheView cache = new AnalysisCacheImpl(new DexIRFactory());
    // getEntrypoints function
    Set<LocatorFlags> flags = HashSetFactory.make();
    flags.add(LocatorFlags.INCLUDE_CALLBACKS);
    flags.add(LocatorFlags.EP_HEURISTIC);
    flags.add(LocatorFlags.CB_HEURISTIC);
    AndroidEntryPointLocator eps = new AndroidEntryPointLocator(flags);
    List<? extends Entrypoint> es = eps.getEntryPoints(cha);
    assert !es.isEmpty();

    AnalysisOptions options = new AnalysisOptions(scope,es);
    options.setReflectionOptions(ReflectionOptions.NONE);
    SSAPropagationCallGraphBuilder cgb = 
            com.ibm.wala.ipa.callgraph.impl.Util.makeZeroCFABuilder(Language.JAVA,options, cache, cha, scope);//for wala-1.5.0
            //com.ibm.wala.ipa.callgraph.impl.Util.makeZeroCFABuilder(options, cache, cha, scope);//for wala-1.4.3
    callGraph = cgb.makeCallGraph(options,null);
    System.out.println(CallGraphStats.getStats(callGraph));
    System.out.println("Time spent ont building CHA and CG:" + (System.currentTimeMillis() - startTime) + "ms");
}
juliandolby commented 6 years ago

I reproduced that failure, and tracked down a buggy bit of code in the Dakvik front end, which I think I fixed. Can you try again with the latest code from the WALA master?

lczxxx123 commented 6 years ago

I reproduced that failure, and tracked down a buggy bit of code in the Dakvik front end, which I think I fixed. Can you try again with the latest code from the WALA master?

Sorry , now I'm trouble with proxy to access google so that it's a little hard for me to build source project with gradle automaticlly, I hope that you will allow me to do it later after i fix my proxy and @you to bother you about the result.

msridhar commented 6 years ago

@juliandolby can you maybe cut a SNAPSHOT release? That will make it easier for @lczxxx123 to test I think. In any case given the number of reports we've gotten we should probably cut a new release once this is confirmed fixed.

juliandolby commented 6 years ago

@lczxxx123 https://github.com/lczxxx123, would a snapshot release help you test? If so, I can make one. If you can fix your proxy quickly, I would like to wait until you confirm this is fixed for you.

On Sep 12, 2018, at 10:09 AM, Manu Sridharan notifications@github.com wrote:

@juliandolby https://github.com/juliandolby can you maybe cut a SNAPSHOT release? That will make it easier for @lczxxx123 https://github.com/lczxxx123 to test I think. In any case given the number of reports we've gotten we should probably cut a new release once this is confirmed fixed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wala/WALA/issues/350#issuecomment-420662207, or mute the thread https://github.com/notifications/unsubscribe-auth/ABk3frpJcp5sZydhQH_xWjD7PF3b9u9fks5uaRW0gaJpZM4WjHpL.

lczxxx123 commented 6 years ago

@juliandolby actually i have been stuck in proxy problem for days and not find a way to fix that (i think there might be some problem with my system environment but not proxy), could you please make one snapshot to help me test if you have time. I'm not in rush because i can still test parts of APKs basing on 1.4.3 .

msridhar commented 6 years ago

@lczxxx123 can you try version 1.5.1-SNAPSHOT and see if it fixes this problem? I just put up a new snapshot build.

msridhar commented 6 years ago

I'm going to close under the assumption this is fixed. @lczxxx123 please re-open if you still see the problem