search

sireum / archived-v2-amandroid

34 stars 16 forks source link

Amandroid Stability #40

Closed CoolDeveloper1 closed 9 years ago

CoolDeveloper1 commented 9 years ago

Hi Fengguo and Team,

Thank you for your hard work on this project. I would like to use Amandroid as a platform to build further analysis, but I have been running into many errors just running the included plugins. I want to make sure Amandroid is a stable platform before I invest significant time into development. I have been testing both the Shortcake stable and dev branches on a Mac OSX 10.9 laptop.

Here is an example run of a CLI plugin that illustrates my concern. I have been testing each of the plugins on a set of 24 APKs, primarily from the Google Play store. This particular run is using the latest version of Amandroid dev.

time ./sireum amandroid staging -o /path/Desktop/amandroid_eval/dev_build/staging/ -m 8 -t DIR /path/Desktop/apks/ 2>&1 | tee ~/Desktop/amandroid_eval/dev_build/staging/out.txt

Command Line Output Total apks: 24

Error: null Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog

Error: null Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog

Error: null Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog

Error: cannot find field: @@แ$･.ι Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog

Error: null Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog

Error: Cannot resolve concrete dispatch! Type:android.content.ContextWrapper Method:setListAdapter:(Landroid/widget/ListAdapter;)V Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog

Error: null Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog

Error: Cannot resolve concrete dispatch! Type:berserker.android.corelib.t Method:remove:(Ljava/lang/String;)Landroid/content/SharedPreferences$Editor; Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog

Error: null Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog

Error: null Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog

Error: null Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog com.digg.widget.DiggAppWidgetProvider result stored! com.nventive.android.auth.OAuthActivity result stored! com.digg.activities.preferences.PreferencesActivityBookmarks result stored! com.digg.activities.StoryDetailsActivity result stored! com.digg.activities.SearchActivity result stored! com.facebook.FacebookActivity result stored! com.digg.activities.MainActivity result stored! com.digg.activities.preferences.PreferencesActivityAccount result stored! com.digg.activities.WebViewActivity result stored! com.digg.widget.DiggWidgetService result stored! com.digg.auth.GoogleAuthActivity result stored! com.digg.activities.preferences.PreferencesActivity result stored! com.digg.activities.DrawerActivity result stored! com.digg.activities.preferences.PreferencesActivityAbout result stored! com.digg.activities.AddSubscriptionActivity result stored!

Error: null Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog com.saurik.substrate.RestartReceiver result stored! com.saurik.substrate.GalleryActivity result stored! com.saurik.substrate.PackageReceiver result stored! com.saurik.substrate.SetupActivity result stored!

Error: null Written: /path/Desktop/amandroid_eval/dev_build/staging/.errorlog

Out of the 24 APKs in this run, very few generated a IDFG or DDG. I am commonly running into Null and Concrete Dispatch errors. I see that other users have opened issues about these errors (see https://github.com/sireum/amandroid/issues/26, https://github.com/sireum/amandroid/issues/31) but I am not aware of a solution. Please correct me if I am wrong.

In your paper you discuss evaluating 753 Google Play store apps. Evaluating so many apps without errors should indicate that the Amandroid platform is stable. However, that is not what I am seeing in my tests and the Issues page.

Are you also encountering these errors? Is my setup wrong somehow? Any comments or suggestions are appreciated.

One suggestion I have for Amandroid - Error logs should not be overwritten when running with --type DIR run. This makes debugging errors more difficult.

fgwei commented 9 years ago

Could you show me the errorlog?

fgwei commented 9 years ago

Oh, I just see that you are running staging... I have not do/update staging for a long time. So that code does not work. And I forget to pull it off from the cli...

fgwei commented 9 years ago

You need that functionality?

CoolDeveloper1 commented 9 years ago

OK that makes sense. No, I don't need staging CLI as long as I can generate the IDFG and DDG through AndroidReachingFactsAnalysis() and InterproceduralDataDependenceAnalysis().

I am also confused by the output of CryptoMisuse CLI.

Command Line Output Total apks: 24 Ignored! Ignored! Ignored!

Error: cannot find field: @@แ$･.ι Written: /Users/edwardtanner/Desktop/amandroid_eval/dev_build/cryptoMisuse/.errorlog Ignored! Ignored! Ignored! Ignored! Ignored! Ignored! Ignored! Ignored! Ignored! Ignored! Ignored! Ignored! Ignored!

What does the the IgnoreException mean?

CoolDeveloper1 commented 9 years ago

My goal is to write plugins, and I would like to use CryptoMisuse as a model to help develop my first plugin. But I want to make sure the plugin and platform are running smoothly before I start.

Another question. From what I understand from the dev guide (http://amandroid.sireum.org/docs/devguide.html) I should implement new plugins in org.sireum.amandroid.security. It looks like I write code to call this plugin in the org.sireum.amandroid.run.security package - is that correct? Is there a central location in code where I can generate the IDFG and DDG once, then use that to call all of the plugins?

Thanks.

CoolDeveloper1 commented 9 years ago

I tried running org.sireum.amandroid.run.security.CryptoMisuse_run in Eclipse with program arguments path/apks/app.apk /path2/cryptoMisuse and received the following error:

Exception in thread "main" java.lang.NullPointerException at scala.collection.mutable.ArrayOps$ofRef$.length$extension(ArrayOps.scala:192) at scala.collection.mutable.ArrayOps$ofRef.length(ArrayOps.scala:192) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:32) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) at org.sireum.util.FileUtil$.listFiles(File.scala:47) at org.sireum.jawa.JawaCodeSource$.preLoad(JawaCodeSource.scala:59) at org.sireum.amandroid.security.AmandroidSocket.preProcess(AmandroidSocket.scala:65) at org.sireum.amandroid.run.security.CryptoMisuse_run$.main(CryptoMisuse_run.scala:73) at org.sireum.amandroid.run.security.CryptoMisuse_run.main(CryptoMisuse_run.scala)

Is this an error with the plugin or the way I am calling it? Essentially the same question as above. Any help to get me started without errors would be greatly appreciated.

fgwei commented 9 years ago

First of all, the the ignore exception means the target apk component does not have crypto related apis. So it ignoring it to save time. If you want to write a plugin, you can follow the writing style of CryptoMisuse_run or Dataleakage_run.

I didn't put enough exception handling code in Amandroid, that is the reason why you feel it is not stable. And the new version will coming soon, that will put more effort on the error handling. The error you are facing above has two possibilities:

  1. your first argument should be the dir of the source apk. so it should be /path/apks
  2. Check and see whether you have "$SIREUM_HOME/apps/amandroid/androidlib/5.0" folder.
CoolDeveloper1 commented 9 years ago

Gotchya, thank you for your response. I did not have "$SIREUM_HOME/apps/amandroid/androidlib/5.0" in the Strawberry stable distribution. The plugin is running correctly when I open Eclipse from the Strawberry dev distribution.