soot-oss / soot

Soot - A Java optimization framework
GNU Lesser General Public License v2.1
2.85k stars 706 forks source link

Simple CallGraph Example for Android App #1472

Open PeterParker369 opened 3 years ago

PeterParker369 commented 3 years ago

Is it possible to create a simple standalone example for how to use actual soot stuff to create call graph for Android applications?

A simple example step-by-step:

  1. How to load APK?
  2. Select class and method to lookup, find all usages. How to correctly set entry point or createDummyMain() (as in old examples)?
  3. Build Call Graph.

I think people are getting confused with versions of jars are used in the project. So me too. Some examples (that can be found in the internet) became old and included methods are not containd in current versions of soot anymore. This is also very confusing

Thanks in advance.

StevenArzt commented 3 years ago

You can use FlowDroid to build the callgraph of an APK:

SetupApplication app = new SetupApplication("Test.apk", "/opt/android-sdk-linux/platforms");
app.constructCallgraph();

Afterwards, you have a Soot instance with a callgraph.

PeterParker369 commented 3 years ago

Thanks Steven (@StevenArzt)

I've read a lot of info and finally succeded. So when starting to analyze an apk i'm getting next info and warnings:

[main] INFO soot.jimple.infoflow.android.callbacks.DefaultCallbackAnalyzer - Running incremental callback analysis for 0 components... [main] INFO soot.jimple.infoflow.android.callbacks.DefaultCallbackAnalyzer - Incremental callback analysis done. [main] INFO soot.jimple.infoflow.android.entryPointCreators.AndroidEntryPointCreator - Creating Android entry point for 350 components... [main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for androidx.fragment.app.FragmentTransition$Callback, because it is an interface and cannot substitute with subclass [main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for androidx.fragment.app.FragmentTransition$Callback, because it is an interface and cannot substitute with subclass [main] INFO soot.jimple.infoflow.android.SetupApplication - Constructing the callgraph...

This info is showed repeteadly like there is a loop. Is it correct? So I have same questions like in issue https://github.com/secure-software-engineering/FlowDroid/issues/135

  1. So are the entry points in created correctly?
  2. Will there be the end of analysis? Will it stop to show this warnings?
  3. Any hints for Options class?

Sorry for bothering. Waiting for reply:)

StevenArzt commented 3 years ago

I'm glad to hear that you succeeded in creating a callgraph.

You don't need to set any Soot options manually, FlowDroid takes care of this. The warnings are expected. FlowDroid tells you about classes that cannot be instantiated as part of the dummy main method, i.e., the new entry point. Strictly speaking, FlowDroid warns you about possible causes of incompleteness in your callgraph. If you find a missing edge that is important, these warnings might tell you where to start looking. However, the FlowDroid callgraph is usually fine in practice, as these abstract classes are mostly corner cases.

The callgraph construction will eventually reach a fixpoint and your call to app.constructCallgraph(); will return. I'm not quite sure what your question is about. Returning from a call doesn't have much to do with the output we write to the log.

PeterParker369 commented 3 years ago

I succeeded to analyze very small test apps. But wasn't able to analyze any real app. I'm trying to build a callgraph for UCWebBrowser application (android:versionName="11.5.0.1015" package="com.UCMobile.intl" platformBuildVersionCode="24") (APK file: https://ufile.io/6hk03uk7). It starts to work correctly (constructCallgraph() method), but then loops. It repeatedly puts the text block below and doesnt stops. So my questions:

  1. How to skip analyzing abstract classes? or maybe add any options?
  2. How long will and must take this 30 Mbyte app analysis? I waited for 30 minutes and it continued texting this messages below, then stopped.

Please try this app. What should be done to get the constructCallgraph() method done? Please @StevenArzt


[main] INFO soot.jimple.infoflow.android.callbacks.DefaultCallbackAnalyzer - Running incremental callback analysis for 0 components...
[main] INFO soot.jimple.infoflow.android.callbacks.DefaultCallbackAnalyzer - Incremental callback analysis done.
[main] INFO soot.jimple.infoflow.android.entryPointCreators.AndroidEntryPointCreator - Creating Android entry point for 137 components...
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.ui.widget.c.am, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.BroadcastReceiverEntryPointCreator - Cannot create valid constructor for com.uc.processmodel.AbstractProcess, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.ui.widget.c.am, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.ui.widget.titlebar.j, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.BroadcastReceiverEntryPointCreator - Cannot create valid constructor for com.uc.processmodel.AbstractProcess, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.ui.widget.c.am, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.browser.bgprocess.bussinessmanager.locksecurity.pattern.c, because it is an interface and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.browser.business.defaultbrowser.f, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.BroadcastReceiverEntryPointCreator - Cannot create valid constructor for com.uc.addon.engine.bw, because it is an interface and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.e.b.h, because it is an interface and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.e.b.g, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.e.b.d, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.ui.widget.c.am, because it is abstract and cannot substitute with subclass
[main] INFO soot.jimple.infoflow.android.SetupApplication - Constructing the callgraph...
xingyueren-qinmu commented 2 years ago

I succeeded to analyze very small test apps. But wasn't able to analyze any real app. I'm trying to build a callgraph for UCWebBrowser application (android:versionName="11.5.0.1015" package="com.UCMobile.intl" platformBuildVersionCode="24") (APK file: https://ufile.io/6hk03uk7). It starts to work correctly (constructCallgraph() method), but then loops. It repeatedly puts the text block below and doesnt stops. So my questions:

  1. How to skip analyzing abstract classes? or maybe add any options?
  2. How long will and must take this 30 Mbyte app analysis? I waited for 30 minutes and it continued texting this messages below, then stopped.

Please try this app. What should be done to get the constructCallgraph() method done? Please @StevenArzt


[main] INFO soot.jimple.infoflow.android.callbacks.DefaultCallbackAnalyzer - Running incremental callback analysis for 0 components...
[main] INFO soot.jimple.infoflow.android.callbacks.DefaultCallbackAnalyzer - Incremental callback analysis done.
[main] INFO soot.jimple.infoflow.android.entryPointCreators.AndroidEntryPointCreator - Creating Android entry point for 137 components...
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.ui.widget.c.am, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.BroadcastReceiverEntryPointCreator - Cannot create valid constructor for com.uc.processmodel.AbstractProcess, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.ui.widget.c.am, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.ui.widget.titlebar.j, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.BroadcastReceiverEntryPointCreator - Cannot create valid constructor for com.uc.processmodel.AbstractProcess, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.ui.widget.c.am, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.browser.bgprocess.bussinessmanager.locksecurity.pattern.c, because it is an interface and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.browser.business.defaultbrowser.f, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.BroadcastReceiverEntryPointCreator - Cannot create valid constructor for com.uc.addon.engine.bw, because it is an interface and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.e.b.h, because it is an interface and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.e.b.g, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.e.b.d, because it is abstract and cannot substitute with subclass
[main] WARN soot.jimple.infoflow.android.entryPointCreators.components.ActivityEntryPointCreator - Cannot create valid constructor for com.uc.framework.ui.widget.c.am, because it is abstract and cannot substitute with subclass
[main] INFO soot.jimple.infoflow.android.SetupApplication - Constructing the callgraph...

Has your problem been solved? I have the same trouble today. Endless loop...

fabianhick commented 11 months ago

I can confirm experiencing this issue with real world apps as well. I wanted to compare the call graph generated by androguard to the one retrieved by FlowDroid, but, unfortunately, it hasn't created one in over an hour for a recent Firefox apk. Memory usage of the JVM was below 5GB consistently so far.

How long is FlowDroid supposed to run, @StevenArzt? androguard's call graph generation completed within a few minutes.