secure-software-engineering / FlowDroid

FlowDroid Static Data Flow Tracker
GNU Lesser General Public License v2.1
1.06k stars 298 forks source link

Questions about using FlowDroid with STUBDROID #108

Open liuyongxiang opened 5 years ago

liuyongxiang commented 5 years ago

Hello~ If I want to use the StubDroid generated method summary when running FlowDroid, do I need to specify the summaries directory (XML files) with the "-t /XXX/summary" command and use "-tw STUBDROID" to enable the STUBDROID? And so what settings should I use to allow FlowDroid to analyze the entire library function? Best wishes!

StevenArzt commented 5 years ago

StubDroid is enabled by default and uses the summaries compiled into the JAR, or from the "summariesManual" directory on disk if you run the program from the source distribution. You can use the command-line option to specify an alternate search directory for summaries. You can use the -t option multiple times to specify multiple directories. Keep in mind to also reference the original summary directory to retain StubDroid's default summaries, or else they will be discarded.

As a shortcut for analyzing the complete library implementation without taking StubDroid summaries into account, you can use the -ff option. It will internally set various options to disable the taint wrappers and some shortcuts, that would otherwise lead to skipped classes.

Note that I'm referring to FlowDroid 2.7 here, i.e., the current "develop" branch. I'm still sorting out a single failing test case, and I'll hope to have a proper release soon.

liuyongxiang commented 5 years ago

Wow~I am very happy to receive your reply. I still want to ask, “Keep in mind to also reference the original summary directory to retain StubDroid's default summaries”, does this sentence refer to the summaries compiled into the JAR?

StevenArzt commented 5 years ago

You would simply reference the "summariesManual" directory on disk, which is located under "soot-infoflow-summaries" inside the FlowDroid source directory. The same files are compiled into the JAR, but it's easier to use them from disk if you also want to include your own XML files as well.

StevenArzt commented 5 years ago

These warnings are not a problem. You can usually safely ignore them unless callbacks are missing from the callgraph. FlowDroid tries to find the declarations of all classes referenced in the layout XML files. These classes are part of the Android platform JAR. Depending on your JAR file, not all support classes may be there. Alternatively, the support library may also be compiled into the APK itself. In that case, it is important that you use the option for merging the dex files (-d).

liuyongxiang commented 5 years ago

When I use FlowDroid 2.7 to analyze APK, some APKs are fine, but some will always report errors during the running process. I am confused. Is this a bug?

[main] WARN soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Running with limited join point abstractions can break context-sensitive path builders [main] INFO soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Looking for sources and sinks... [main] INFO soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Source lookup done, found 112 sources and 675 sinks. [FlowDroid] ERROR heros.solver.CountingThreadPoolExecutor - Worker thread execution failed: null java.lang.NullPointerException at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:210) at com.google.common.cache.LocalCache.get(LocalCache.java:3936) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824) at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4830) at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getOrCreateUnitGraph(AbstractJimpleBasedICFG.java:116) at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getPredsOf(AbstractJimpleBasedICFG.java:231) at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getPredsOf(AbstractJimpleBasedICFG.java:1) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getPredsOfCallAt(InfoflowCFG.java:227) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getPredsOfCallAt(InfoflowCFG.java:1) at soot.jimple.toolkits.ide.icfg.BackwardsInterproceduralCFG.getReturnSitesOfCallAt(BackwardsInterproceduralCFG.java:61) at soot.jimple.toolkits.ide.icfg.BackwardsInterproceduralCFG.getReturnSitesOfCallAt(BackwardsInterproceduralCFG.java:1) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getReturnSitesOfCallAt(InfoflowCFG.java:207) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getReturnSitesOfCallAt(InfoflowCFG.java:1) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver.processExit(IFDSSolver.java:504) at soot.jimple.infoflow.solver.fastSolver.InfoflowSolver.processExit(InfoflowSolver.java:127) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver$PathEdgeProcessingTask.run(IFDSSolver.java:738) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Exception in thread "FlowDroid" java.lang.NullPointerException at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:210) at com.google.common.cache.LocalCache.get(LocalCache.java:3936) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824) at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4830) at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getOrCreateUnitGraph(AbstractJimpleBasedICFG.java:116) at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getPredsOf(AbstractJimpleBasedICFG.java:231) at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getPredsOf(AbstractJimpleBasedICFG.java:1) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getPredsOfCallAt(InfoflowCFG.java:227) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getPredsOfCallAt(InfoflowCFG.java:1) at soot.jimple.toolkits.ide.icfg.BackwardsInterproceduralCFG.getReturnSitesOfCallAt(BackwardsInterproceduralCFG.java:61) at soot.jimple.toolkits.ide.icfg.BackwardsInterproceduralCFG.getReturnSitesOfCallAt(BackwardsInterproceduralCFG.java:1) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getReturnSitesOfCallAt(InfoflowCFG.java:207) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getReturnSitesOfCallAt(InfoflowCFG.java:1) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver.processExit(IFDSSolver.java:504) at soot.jimple.infoflow.solver.fastSolver.InfoflowSolver.processExit(InfoflowSolver.java:127) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver$PathEdgeProcessingTask.run(IFDSSolver.java:738) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) [main] INFO soot.jimple.infoflow.memory.MemoryWarningSystem - Shutting down the memory warning system... [main] ERROR soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow - Exception during data flow analysis java.lang.RuntimeException: There were exceptions during IFDS analysis. Exiting. at soot.jimple.infoflow.solver.fastSolver.IFDSSolver.runExecutorAndAwaitCompletion(IFDSSolver.java:248) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver.awaitCompletionComputeValuesAndShutdown(IFDSSolver.java:215) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver.solve(IFDSSolver.java:187) at soot.jimple.infoflow.Infoflow.runAnalysis(Infoflow.java:486) at soot.jimple.infoflow.Infoflow.runAnalysis(Infoflow.java:240) at soot.jimple.infoflow.android.SetupApplication$InPlaceInfoflow.runAnalysis(SetupApplication.java:1206) at soot.jimple.infoflow.android.SetupApplication.processEntryPoint(SetupApplication.java:1457) at soot.jimple.infoflow.android.SetupApplication.runInfoflow(SetupApplication.java:1389) at soot.jimple.infoflow.android.SetupApplication.runInfoflow(SetupApplication.java:1335) at soot.jimple.infoflow.cmd.MainClass.run(MainClass.java:334) at soot.jimple.infoflow.cmd.MainClass.main(MainClass.java:232) Caused by: java.lang.NullPointerException at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:210) at com.google.common.cache.LocalCache.get(LocalCache.java:3936) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824) at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4830) at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getOrCreateUnitGraph(AbstractJimpleBasedICFG.java:116) at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getPredsOf(AbstractJimpleBasedICFG.java:231) at soot.jimple.toolkits.ide.icfg.AbstractJimpleBasedICFG.getPredsOf(AbstractJimpleBasedICFG.java:1) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getPredsOfCallAt(InfoflowCFG.java:227) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getPredsOfCallAt(InfoflowCFG.java:1) at soot.jimple.toolkits.ide.icfg.BackwardsInterproceduralCFG.getReturnSitesOfCallAt(BackwardsInterproceduralCFG.java:61) at soot.jimple.toolkits.ide.icfg.BackwardsInterproceduralCFG.getReturnSitesOfCallAt(BackwardsInterproceduralCFG.java:1) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getReturnSitesOfCallAt(InfoflowCFG.java:207) at soot.jimple.infoflow.solver.cfg.InfoflowCFG.getReturnSitesOfCallAt(InfoflowCFG.java:1) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver.processExit(IFDSSolver.java:504) at soot.jimple.infoflow.solver.fastSolver.InfoflowSolver.processExit(InfoflowSolver.java:127) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver$PathEdgeProcessingTask.run(IFDSSolver.java:738) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) [main] INFO soot.jimple.infoflow.android.SetupApplication - Found 0 leaks [FlowDroid] ERROR heros.solver.CountingThreadPoolExecutor - Worker thread execution failed: null java.lang.NullPointerException at soot.jimple.infoflow.problems.rules.SinkPropagationRule.propagateReturnFlow(SinkPropagationRule.java:172) at soot.jimple.infoflow.problems.rules.PropagationRuleManager.applyReturnFlowFunction(PropagationRuleManager.java:235) at soot.jimple.infoflow.problems.InfoflowProblem$1$3.computeTargetsInternal(InfoflowProblem.java:551) at soot.jimple.infoflow.problems.InfoflowProblem$1$3.computeTargets(InfoflowProblem.java:512) at soot.jimple.infoflow.solver.fastSolver.InfoflowSolver.computeReturnFlowFunction(InfoflowSolver.java:79) at soot.jimple.infoflow.solver.fastSolver.InfoflowSolver.computeReturnFlowFunction(InfoflowSolver.java:1) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver.processExit(IFDSSolver.java:507) at soot.jimple.infoflow.solver.fastSolver.InfoflowSolver.processExit(InfoflowSolver.java:127) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver$PathEdgeProcessingTask.run(IFDSSolver.java:738) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Exception in thread "FlowDroid" java.lang.NullPointerException at soot.jimple.infoflow.problems.rules.SinkPropagationRule.propagateReturnFlow(SinkPropagationRule.java:172) at soot.jimple.infoflow.problems.rules.PropagationRuleManager.applyReturnFlowFunction(PropagationRuleManager.java:235) at soot.jimple.infoflow.problems.InfoflowProblem$1$3.computeTargetsInternal(InfoflowProblem.java:551) at soot.jimple.infoflow.problems.InfoflowProblem$1$3.computeTargets(InfoflowProblem.java:512) at soot.jimple.infoflow.solver.fastSolver.InfoflowSolver.computeReturnFlowFunction(InfoflowSolver.java:79) at soot.jimple.infoflow.solver.fastSolver.InfoflowSolver.computeReturnFlowFunction(InfoflowSolver.java:1) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver.processExit(IFDSSolver.java:507) at soot.jimple.infoflow.solver.fastSolver.InfoflowSolver.processExit(InfoflowSolver.java:127) at soot.jimple.infoflow.solver.fastSolver.IFDSSolver$PathEdgeProcessingTask.run(IFDSSolver.java:738) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) [FlowDroid Timeout Watcher] INFO soot.jimple.infoflow.memory.FlowDroidTimeoutWatcher - FlowDroid timeout watcher terminated

StevenArzt commented 5 years ago

Please provide a link to an APK with which the problem occurs.