After constructcallgraph changing Jimple body in PackManager

[jyotigajrani]

I have constructed callgraph using

SetupApplication ap = new SetupApplication(androidJarPath, apkPath); ap.setCallbackFile("AndroidCallbacks.txt"); ap.constructCallgraph(); InfoflowCFG icfg=new InfoflowCFG(); we perform some processing on result of this icfg.

But after this, when i do internalTransform, it misses some classes. To avoid this, I re-initialize soot options along with soot.G.reset(); So now we are able to get all the classes in internalTransform. But we are facing other issue that the Jimple body gets changed....The units which were present earlier with some Local $r2 now gets completely changed... we want to access the same body that was while processing icfg control paths. How can we avoid changing body even after re-initialization.

@ericbodden Please help...we are stuck due to the issue. Thanks

[StevenArzt]

FlowDroid excludes some classes for performance reasons, because these classes are not required for the data flow analysis. To avoid this problem, you can initialize Soot on your own and run FlowDroid's callgraph construction on your existing Soot instance. Have a look at the SootIntegrationMode enum inside the InfoflowAndroidConfiguration class. It's a bit tricky, though.

Roughly, the steps are as follows: Configure Soot, call Scene.v().loadNecessaryClasses, configure FlowDroid to run inside the existing Soot instance and then call constructCallgraph().

[jyotigajrani]

Thanks for answer. Please provide some more detail one: configure FlowDroid to run inside the existing Soot instance

Any link........or some code hints please. It will help...

[umang78]

Sir I am not clear with sentence "create an instance of soot" please help to clear this.

@ericbodden

[umang78]

Sir, As per the steps told by you we are doing this please check if any correction required.

SetupApplication analyzer = null; InfoflowAndroidConfiguration config = new InfoflowAndroidConfiguration(); config.getAnalysisFileConfig().setAndroidPlatformDir(androidJarPath); config.getAnalysisFileConfig().setTargetAPKFile(apkPath); analyzer = new SetupApplication(config); soot.options.Options.v().set_whole_program(true); soot.options.Options.v().set_allow_phantom_refs(true); soot.options.Options.v().set_prepend_classpath(true); soot.options.Options.v().set_force_android_jar(androidJarPath); soot.options.Options.v().set_src_prec(Options.src_prec_apk); soot.options.Options.v().set_android_jars(androidJarPath); soot.options.Options.v().force_overwrite(); soot.options.Options.v().set_process_dir(Collections.singletonList(apkPath)); soot.options.Options.v().set_output_format(Options.output_format_dex);
soot.options.Options.v().set_force_overwrite(true); Scene.v().addBasicClass("java.io.PrintStream",SootClass.SIGNATURES); soot.options.Options.v().setPhaseOption("cg.spark", "on");

    Scene.v().loadNecessaryClasses();

    config.setSootIntegrationMode(InfoflowAndroidConfiguration.SootIntegrationMode.UseExistingInstance);

IInfoflowConfig sootConfig = new SootConfigForAndroid();
    sootConfig.setSootOptions(soot.options.Options.v(), config);
    analyzer = new SetupApplication(config);
    analyzer.setSootConfig(sootConfig);
    analyzer.constructCallgraph();
    InfoflowCFG icfg=new InfoflowCFG();

Thank you @StevenArzt @ericbodden

[jyotigajrani]

After this we are able to get all classes inside Packmanager but now when we access icfg inside the PackManager, it is null. Earlier the icfg was not Null. @StevenArzt Please suggest how can we get icfg if we use this way of constructing callgraph.

Regards

[jyotigajrani]

@StevenArzt We were able to get all classes inside internalTransform with this approach. We have done few instrumentations also without any error. But when we execute the app, it unfortunately stop. While the same thing when we do with normal method without using icfg, we are able to get executable app. The options are set as follows: `SetupApplication analyzer = null; InfoflowAndroidConfiguration config = new InfoflowAndroidConfiguration(); config.getAnalysisFileConfig().setAndroidPlatformDir(androidJarPath); config.getAnalysisFileConfig().setTargetAPKFile(apkPath); soot.options.Options.v().set_whole_program(true); soot.options.Options.v().set_allow_phantom_refs(true); soot.options.Options.v().set_prepend_classpath(true); soot.options.Options.v().set_src_prec(Options.src_prec_apk); soot.options.Options.v().set_android_jars(androidJarPath); soot.options.Options.v().force_overwrite(); soot.options.Options.v().set_process_dir(Collections.singletonList(apkPath)); soot.options.Options.v().set_output_format(Options.output_format_dex);
soot.options.Options.v().setPhaseOption("cg.spark", "on"); soot.options.Options.v().set_no_bodies_for_excluded(true); Scene.v().addBasicClass("java.io.PrintStream",SootClass.SIGNATURES); Scene.v().loadNecessaryClasses(); config.setSootIntegrationMode(InfoflowAndroidConfiguration.SootIntegrationMode.UseExistingInstance); IInfoflowConfig sootConfig = new SootConfigForAndroid(); sootConfig.setSootOptions(soot.options.Options.v(), config); analyzer = new SetupApplication(config); analyzer.setCallbackFile("AndroidCallbacks.txt"); analyzer.setSootConfig(sootConfig); analyzer.constructCallgraph(); if (icfgFactory == null) { DefaultBiDiICFGFactory factory = new DefaultBiDiICFGFactory(); factory.setIsAndroid(androidJarPath != null && !androidJarPath.isEmpty()); icfgFactory = factory; }

final InfoflowCFG icfg = (InfoflowCFG) icfgFactory.buildBiDirICFG(config.getCallgraphAlgorithm(), config.getEnableExceptionTracking());`

The error in logcat when running app is W/dalvikvm( 1320): Unable to resolve superclass of Lcom/example/sixactivity/HomeActivity; (10) W/dalvikvm( 1320): Link of class 'Lcom/example/sixactivity/HomeActivity;' failed D/AndroidRuntime( 1320): Shutting down VM W/dalvikvm( 1320): threadid=1: thread exiting with uncaught exception (group=0x40a122a0) E/AndroidRuntime( 1320): FATAL EXCEPTION: main E/AndroidRuntime( 1320): java.lang.RuntimeException: Unable to instantiate activity ComponentInfo{com.example.sixactivity/com.example.sixactivity.HomeActivity}: java.lang.ClassNotFoundException: com.example.sixactivity.HomeActivity E/AndroidRuntime( 1320): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1983) E/AndroidRuntime( 1320): at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2084) E/AndroidRuntime( 1320): at android.app.ActivityThread.access$600(ActivityThread.java:130) E/AndroidRuntime( 1320): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1195) E/AndroidRuntime( 1320): at android.os.Handler.dispatchMessage(Handler.java:99) E/AndroidRuntime( 1320): at android.os.Looper.loop(Looper.java:137) E/AndroidRuntime( 1320): at android.app.ActivityThread.main(ActivityThread.java:4745) E/AndroidRuntime( 1320): at java.lang.reflect.Method.invokeNative(Native Method) E/AndroidRuntime( 1320): at java.lang.reflect.Method.invoke(Method.java:511) E/AndroidRuntime( 1320): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:786) E/AndroidRuntime( 1320): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553) E/AndroidRuntime( 1320): at dalvik.system.NativeStart.main(Native Method) E/AndroidRuntime( 1320): Caused by: java.lang.ClassNotFoundException: com.example.sixactivity.HomeActivity E/AndroidRuntime( 1320): at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:61) E/AndroidRuntime( 1320): at java.lang.ClassLoader.loadClass(ClassLoader.java:501) E/AndroidRuntime( 1320): at java.lang.ClassLoader.loadClass(ClassLoader.java:461) E/AndroidRuntime( 1320): at android.app.Instrumentation.newActivity(Instrumentation.java:1053) E/AndroidRuntime( 1320): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1974) E/AndroidRuntime( 1320): ... 11 more

Please check and suggest what may be the reason of app stopping after installation. Is some problem in setting options? Please reply with some hints. Thanks

[StevenArzt]

Have you tried to disable the code rewriting options of FlowDroid? The code would be

config.setCodeEliminationMode(CodeEliminationMode.NoCodeElimination)

In its default configuration, FlowDroid performs various optimizations to the code that are intended to retain the original semantics of the app with regard to static data flow analysis, but not necessarily with regard to execution.

[jyotigajrani]

Thanks @StevenArzt for reply. I added this option also and now the options are like this:

`private static void setOptions() { InfoflowAndroidConfiguration config = new InfoflowAndroidConfiguration();

config.getAnalysisFileConfig().setAndroidPlatformDir(androidJarPath);

config.getAnalysisFileConfig().setTargetAPKFile(apkPath);

config.setSootIntegrationMode(InfoflowAndroidConfiguration.SootIntegrationMode.UseExistingInstance);

config.setCodeEliminationMode(CodeEliminationMode.NoCodeElimination);

SetupApplication analyzer = null;

analyzer = new SetupApplication(androidJarPath, apkPath);

analyzer.setCallbackFile("AndroidCallbacks.txt");

analyzer.setSootConfig(new IInfoflowConfig() {

public void setSootOptions(Options options, InfoflowConfiguration config) {

options.set_whole_program(true);

options.set_allow_phantom_refs(true);

options.set_prepend_classpath(true);

options.set_src_prec(Options.src_prec_apk);

options.set_android_jars(androidJarPath);

options.force_overwrite();

options.set_process_dir(Collections.singletonList(apkPath));

options.set_output_format(Options.output_format_dex);

options.set_force_overwrite(true);

options.setPhaseOption("cg.spark", "on");

options.set_process_multiple_dex(true);

((InfoflowAndroidConfiguration) config).getAnalysisFileConfig().setAndroidPlatformDir(androidJarPath);

((InfoflowAndroidConfiguration) config).getAnalysisFileConfig().setTargetAPKFile(apkPath);

((InfoflowAndroidConfiguration) config).setSootIntegrationMode(InfoflowAndroidConfiguration.SootIntegrationMode.UseExistingInstance); } });

Scene.v().addBasicClass("java.io.PrintStream",SootClass.SIGNATURES);

Scene.v().loadNecessaryClasses();

analyzer.constructCallgraph();
} ` and even we removed all code from internalTransform. But still the generated apk is not executable. But one thing we noticed that earlier we were not setting option - options.set_validate(true); And this time when we add this option it started giving exception and could not generate apk. The exception is as :

(<dummyMainClass: com.example.dell.quizzy.Instructions dummyMainMethod_Instructions(android.content.Intent)>) no defs for value: $r0! at soot.validation.UsesValidator.validate(UsesValidator.java:97) at soot.Body.validate(Body.java:253) at soot.jimple.JimpleBody.validate(JimpleBody.java:112) at soot.jimple.JimpleBody.validate(JimpleBody.java:98) at CodeOfNOP$1.internalTransform(CodeOfNOP.java:71) at soot.BodyTransformer.transform(BodyTransformer.java:51) at soot.Transform.apply(Transform.java:120) at soot.BodyPack.internalApply(BodyPack.java:50) at soot.Pack.apply(Pack.java:125) at soot.PackManager.runBodyPacks(PackManager.java:1019) at soot.PackManager.access$000(PackManager.java:146) at soot.PackManager$1.run(PackManager.java:664) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

The internalTransform does not contain any instrumentation code so we think the problem is in setting options. And If we explicitly remove dummyMainClass from Scene, the exception goes. Scene.v().removeClass(Scene.v().getSootClass("dummyMainClass"));

Please check options and suggest what can we try further for getting an executable app.

Thanks @StevenArzt Jyoti

[sunxiaobiu]

I encountered a similar problem caused by the change of ICFG and I use the following steps to solve it: `
//step1. SetupApplication analyser = new SetupApplication(forceAndroidJar, apkPath); Scene.v().loadNecessaryClasses(); analyser.constructCallgraph(); GlobalRef.infoflowCFG = new InfoflowCFG();

    //step2.
    SetupApplication setupApplication = new SetupApplication(androidJars,
            fileName);

    // Find the taint wrapper file
    File taintWrapperFile = new File("EasyTaintWrapperSource.txt");
    if (!taintWrapperFile.exists())
        taintWrapperFile = new File("res/EasyTaintWrapperSource.txt");

    // Make sure to apply the settings before we calculate entry points
    if (configCallback != null)
        configCallback.configureAnalyzer(setupApplication.getConfig());

    setupApplication.setTaintWrapper(new EasyTaintWrapper(taintWrapperFile));
    setupApplication.getConfig().setEnableArraySizeTainting(true);

    if (iccModel != null && iccModel.length() > 0) {
        setupApplication.getConfig().getIccConfig().setIccModel(iccModel);
    }

    //set multi-dex
    setupApplication.getConfig().setMergeDexFiles(true);
    setupApplication.getConfig().setWriteOutputFiles(true);

    setupApplication.getConfig().setCodeEliminationMode(InfoflowConfiguration.CodeEliminationMode.NoCodeElimination);
    setupApplication.getConfig().setSootIntegrationMode(InfoflowConfiguration.SootIntegrationMode.UseExistingInstance);

    setupApplication.runInfoflow(destinationPath);`

[StevenArzt]

If you use Soot to write out a new APK file, you need to remove the dummy main method. FlowDroid needs to create the dummy main method to seed the callgraph generation. However, this code is not meant to be executable. It is simply equivalent to the real Android framework in terms of callgraph construction. Therefore, this code may totally fail at runtime. Executing this code serves no purpose.

After FlowDroid is done (callgraph generation, potentially data flow analysis if you need it), you can remove the generated classes methods again. That might be a bit tricky, so I added a new method SetupApplication.removeSimulatedCodeElements() to get rid of all the simulated code elements. Call that method before writing out the files using the PackManager. Note that I haven't really tested the new method in your use case, so tell me if something's wrong.