Exception while executing ic3.jar

[luzhujiu]

Hi @JordanSamhi,

I'm using this ic3.jar to generate the ICC model for IccTA with cmd

java -jar ic3.jar -a myapp.apk -cp ~/Library/Android/sdk/platforms/ -db cc.properties

But I get an exception as follows:

[Call Graph] For information on where the call graph may be incomplete, use the verbose option to the cg phase. [Spark] Pointer Assignment Graph in 0.1 seconds. [Spark] Type masks in 0.0 seconds. [Spark] Pointer Graph simplified in 0.0 seconds. [Spark] Propagation in 6.5 seconds. [Spark] Solution found in 6.5 seconds. Exception in thread "main" java.lang.RuntimeException: Could not find method at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:364) at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:363) at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClassInterfaceCallbacks(AnalyzeJimpleClass.java:388) at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClass(AnalyzeJimpleClass.java:320) at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeMethodForCallbackRegistrations(AnalyzeJimpleClass.java:245) at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeRechableMethods(AnalyzeJimpleClass.java:184) at soot.jimple.infoflow.android.AnalyzeJimpleClass.access$200(AnalyzeJimpleClass.java:64) at soot.jimple.infoflow.android.AnalyzeJimpleClass$1.internalTransform(AnalyzeJimpleClass.java:127) at soot.SceneTransformer.transform(SceneTransformer.java:39) at soot.Transform.apply(Transform.java:90) at soot.ScenePack.internalApply(ScenePack.java:40) at soot.Pack.apply(Pack.java:116) at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:144) at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146) at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64) at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73) at edu.psu.cse.siis.ic3.Main.main(Main.java:13)

I tried two apps with SDK 29&30 and both end up with this exception. Do you have any insight into the problem here? Thanks!

[JordanSamhi]

Hi @dongluzhujiu,

Could you please provide the APKs for which it generates this exception, I will try to replicate the issue and debug it.

[luzhujiu]

Hi @JordanSamhi ,

this is my toy app: sampleapp.apk I hope the thing I missed is not so stupid :P

[yokotayokota]

Hello @JordanSamhi ,

I had been looking for ic3 that works without Dare and I eventually found it in your site. I express my appreciation for your work.

It worked fine with a simple apk made by my self, but with some apks downloaded from Google Play it failed and left some errors. As an example, the error with YouTube apk (v14.25.57) is at the end of this post 1. The error with YouTube apk (v16.37.36) is at the end of this post 2.

Do you have any insight into these errors?

Best regards,

*1 the error with YouTube apk (v14.25.57)
----------------------------
[Spark] Solution found in 24.1 seconds.
Exception in thread "main" java.lang.RuntimeException: Could not find method
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:364)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:363)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.getMethodFromHierarchyEx(AnalyzeJimpleClass.java:363)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClassInterfaceCallbacks(AnalyzeJimpleClass.java:388)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClassInterfaceCallbacks(AnalyzeJimpleClass.java:382)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeClass(AnalyzeJimpleClass.java:320)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeMethodForCallbackRegistrations(AnalyzeJimpleClass.java:245)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.analyzeRechableMethods(AnalyzeJimpleClass.java:184)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass.access$200(AnalyzeJimpleClass.java:64)
        at soot.jimple.infoflow.android.AnalyzeJimpleClass$1.internalTransform(AnalyzeJimpleClass.java:127)
        at soot.SceneTransformer.transform(SceneTransformer.java:39)
        at soot.Transform.apply(Transform.java:90)
        at soot.ScenePack.internalApply(ScenePack.java:40)
        at soot.Pack.apply(Pack.java:116)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:144)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)
----------------------------

*2 the error with YouTube apk (v16.37.36)
----------------------------
Warning: malformed Manifest file: action at depth 3
Error when looking for XML resource files in apk /mnt/c/Users/s.yokota/ApkProjects/YouTube_v16.37.36/APK/YouTube.apk: java.lang.RuntimeException: Unknown entry type
java.lang.RuntimeException: Unknown entry type
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readResourceHeader(ARSCFileParser.java:1141)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:978)
        at soot.jimple.infoflow.android.resources.ARSCFileParser$1.handleResourceFile(ARSCFileParser.java:966)
        at soot.jimple.infoflow.android.resources.AbstractResourceParser.handleAndroidResourceFiles(AbstractResourceParser.java:49)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:959)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:113)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)
Exception in thread "main" java.lang.RuntimeException: Unknown entry type
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readResourceHeader(ARSCFileParser.java:1141)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:978)
        at soot.jimple.infoflow.android.resources.ARSCFileParser$1.handleResourceFile(ARSCFileParser.java:966)
        at soot.jimple.infoflow.android.resources.AbstractResourceParser.handleAndroidResourceFiles(AbstractResourceParser.java:49)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:959)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:113)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)
----------------------------

[JordanSamhi]

Hi @yokotayokota,

Could you share the apk you used ?

[yokotayokota]

Hello @JordanSamhi ,

Thank you for your quick reply. I'm trying to use FlowDroid with ICC (i.e. -im parameter) for vulnerability asessment of Android apps.

These are apks that left errors:

[YouTube_v16.37.36.apk.tar.gz] (https://github.com/JordanSamhi/Tools/files/7244903/YouTube_v16.37.36.apk.tar.gz)

Sorry, I can not attach the other apk (YouTube_v14.25.57.apk.tar.gz). I do not know why. If you do not mind, I will send it to you by e-mail directly

[yokotayokota]

Hello @JordanSamhi,

The reason why I could not attach the apk yesterday is that the size of it is too large. I split it into 2 pieces I am sorry to bother you but could you follow the instruction below:

1. $ tar zxvf 00_YouTube_v14.25.57.apk.tar.gz
2. $ tar zxvf 01_YouTube_v14.25.57.apk.tar.gz
3. $ cat YouTube_v14.25.57.apk_00 YouTube_v14.25.57.apk_01 > YouTube_v14.25.57.apk

00_YouTube_v14.25.57.apk.tar.gz 01_YouTube_v14.25.57.apk.tar.gz

[yokotayokota]

Hello @JordanSamhi,

I made a tiny app test_simple.apk.tar.gz and ic3 left the same error as YouTube_v16.37.36.apk. (i.e. *2 error of my previous comment )

I hope this tiny app will help your investigation.

ic3 error of this app
-----------------------------
$ java -jar ic3.jar  -a /mnt/c/Users/s.yokota/ApkProjects/test_simple/APK/app-debug.apk  -cp /mnt/c/Users/s.yokota/AppData/Local/Android/Sdk/platforms -protobuf /mnt/c/Users/s.yokota/ApkProjects/test_simple/ic3ed/
Error when looking for XML resource files in apk /mnt/c/Users/s.yokota/ApkProjects/test_simple/APK/app-debug.apk: java.lang.RuntimeException: Unknown entry type
java.lang.RuntimeException: Unknown entry type
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readResourceHeader(ARSCFileParser.java:1141)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:978)
        at soot.jimple.infoflow.android.resources.ARSCFileParser$1.handleResourceFile(ARSCFileParser.java:966)
        at soot.jimple.infoflow.android.resources.AbstractResourceParser.handleAndroidResourceFiles(AbstractResourceParser.java:49)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:959)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:113)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)
Exception in thread "main" java.lang.RuntimeException: Unknown entry type
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.readResourceHeader(ARSCFileParser.java:1141)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:978)
        at soot.jimple.infoflow.android.resources.ARSCFileParser$1.handleResourceFile(ARSCFileParser.java:966)
        at soot.jimple.infoflow.android.resources.AbstractResourceParser.handleAndroidResourceFiles(AbstractResourceParser.java:49)
        at soot.jimple.infoflow.android.resources.ARSCFileParser.parse(ARSCFileParser.java:959)
        at edu.psu.cse.siis.ic3.SetupApplication.calculateSourcesSinksEntrypoints(SetupApplication.java:113)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:146)
        at edu.psu.cse.siis.ic3.Ic3Analysis.initializeAnalysis(Ic3Analysis.java:64)
        at edu.psu.cse.siis.coal.Analysis.performAnalysis(Analysis.java:73)
        at edu.psu.cse.siis.ic3.Main.main(Main.java:13)

[JordanSamhi]

Dear @yokotayokota,

I stumbled upon the same error, after investigating, I found that the problem seems to come from Flowdroid (which IC3 relies on), I cannot devote too much time on this, I am sorry. I encourage you to debug and try to find a temporary fix until it is officially fixed.

[yokotayokota]

Dear @JordanSamhi,

Thank you for your investigation. Thank you for your time.

Could you tell me who should I ask for help? Is there anyone who uses ICC static analysis of Android apps recently?

In addition, I would like you to show me how you built your ic3.jar and its sources.

I am willing to wait for your reply. Please do not overdo.

[JordanSamhi]

Concerning Flowdroid, you can continue the discussion in secure-software-engineering/FlowDroid#386 and show the error you had with IC3 above, you can see that it comes from Flowdroid (soot.jimple.infoflow.android.resources.ARSCFileParser.readEntryTable(ARSCFileParser.java:1341)).

Nevertheless, I have tried with apps that I have from other xps, I did not have your problem and was able to run ic3, get the model and launch flowdroid with the -im option.

If you want ready-to-use tools and scripts to perform ICC, you can have a look to this repository : https://github.com/JordanSamhi/RAICC (go to artefacts folder).

To build ic3 : git clone https://github.com/siis/ic3.git cd ic3 mvn clean package -P standalone

[yokotayokota]

Dear @JordanSamhi,

Thank you very much for your suggestions. As you say, I will post the errors to FlowDroid #386. In addition, I will try your RAICC.

May I ask you how you got apks that you succeeded ic3 + FlowDroid -im with? I am wondering if these apks are built on the old environment? If so, FlowDroid does not support apks that are built on recent environment(e.g. my Android Studio is 4.1.1).

Best regards,

[JordanSamhi]

I must say that the apks I tested are old apks, you can find it here: https://github.com/secure-software-engineering/DroidBench/tree/iccta/apk

[yokotayokota]

Good morning @JordanSamhi, (Is it morning there now?)

They are too old. I am sorry.

Could you please allow me to ask you one more question? Does ic3 detect registerReceiver and write out to a model file? If not, is the information in the model file that we get from ic3 essentially as the same as the information in AndroidManifest.xml?

Best regards,

[JordanSamhi]

To allow ic3 to write to a model file, you have to use the -protobuf PATH_TO_MODEL_FILE option. You will not have the same information as there is in the AndroidManifest, since it has to do with data propagation and control flow.

[yokotayokota]

Thank you @JordanSamhi,

Now I really understand. First of all, I need to look at the results using these old apks. Secondly, I make sure that FlowDroid possibly doesn't fit the modern apks.

[yokotayokota]

Hello @JordanSamhi,

I am sorry to bother you. Where are sources of these apks? I want understand these situations deeply but I can not really recognize the internals of apks just only reading Readme sentences.
Did you decompile each apk ?

[JordanSamhi]

I do not have the sources and do not know how to find them, except if you directly ask https://github.com/lilicoding who developed them I believe. Or you can use simple tools like https://github.com/skylot/jadx do that you can have aa certain representation of the source code that represents the apps. Those are simple apps, so jadx output should be close to the reality.

[yokotayokota]

Thank you @JordanSamhi, I know jadx well. I can decompile each apk one by one.

I feel that a bit of lack of robustness is one of FlowDroid's issues. Through a few weeks struggle, FlowDroid seems to be full of sense and sensitivity.

I really appreciate your great help.

[yokotayokota]

Hello @JordanSamhi,

I installed an old version of Android Studio, and it works fine. My test apks built on this environment got results from your ic3+FlowDroid with -im option. While waiting for you, temporarily I will do several test using this environment.

I do no want to rush you. Take your time.

[tyousyuka]

HELLO,@JordanSamhi 1, mvn clean install:install-file -Dfile=libs/coal-all-0.1.7.jar -DgroupId=edu.psu.cse.siis -DartifactId=coal -Dversion=0.1.7 -Dpackaging=jar 2, mvn clean package -P standalone

3, java -jar target/ic3-0.2.1-full.jar -a ActivityCommunication1.apk -cp D:/Android/SDK/platforms -protobuf test

what is the reason? Thank you!!!

[JordanSamhi]

Are you using java 8 ?

[tyousyuka]

@JordanSamhi

yes.

which jdk version does it need?

Are you using java 8 ?

[JordanSamhi]

Please use the ic3.jar in : https://github.com/JordanSamhi/Tools

[yokotayokota]

@JordanSamhi

Hello. Good to see you again. What is the difference between the current ic3.jar and the previous one?

[JordanSamhi]

the ic3.jar in the Tool repository is just a compiled version of this one.

[yokotayokota]

@JordanSamhi

Thank you for your quick reply! So, is this ic3.jar the same one I tried two years ago?

[yokotayokota]

@JordanSamhi

I mean, can I use this ic3.jar to analyse apks built with modern android libraries?

[JordanSamhi]

For some app it can crash, I recommend you to try.

[tyousyuka]

@JordanSamhi HELLO，do you have any time to update the libs of this ic3， for example : soot，flowdroid. And support for the latest android.jar of Android platform

For some app it can crash, I recommend you to try.

[JordanSamhi]

I wanted to do that 2 years ago but it turns out that IC3 was developed with a very old version of FlowDroid meaning that updating it would take a lot of time so I gave up because I am busy. I might consider doing that this year.

[yokotayokota]

Hello @JordanSamhi

I might consider doing that this year.

This is good news!

Of course I know that you don't have responsibility to support AndroidX. But I will be happy if you let me know when you did it. Or I was wondering if you would tell me how to do ICC analysis without ic3.jar for modern apks built with AndroidX .

Best wishes,

[JordanSamhi]

I would let you know!

If you do not want to use IC3, you would have to come up with your own approach and implement it. For instance, constant propagation, etc.