search

ucr-riple / AndroidSlicer

AndroidSlicer is a dynamic slicing tool, useful for a variety of tasks, from testing to debugging to security.
Other
13 stars 4 forks source link

GC overhead limit exceeded when running Instrumenter from source code #2

Open ChunghaSung opened 5 years ago

ChunghaSung commented 5 years ago

Hi, I tried to compile and run "instrumenter" from the source code (instead of running instrumenter.sh).

By setting Instrumenter's main as a default function, I used this arguments. `` com.twitter.android -w -allow-phantom-refs -process-multiple-dex -force-android-jar

/android-28/android.jar -src-prec apk -output-format dex -process-dir /AndroidSlicer/tool/Twitter_v7.93.2-release.50_apkpure.com.apk `` The app I used is from Google Play Store for version 7.93.2 After that, there is no error for a while, and it ran for like 5 minutes. At the end, the program dies because of GC I found that the paper used Twitter as a benchmark. I am curious how to instrument it. I tried it with a small example, it could be done. It would be really appreciated if you help me to figure this out. Below is the error message. Soot started on Sun Jul 28 16:59:22 PDT 2019 [Thread-9] ERROR heros.solver.CountingThreadPoolExecutor - Worker thread execution failed: GC overhead limit exceeded java.lang.OutOfMemoryError: GC overhead limit exceeded at soot.toDex.ConstantVisitor.caseStringConstant(ConstantVisitor.java:86) at soot.jimple.StringConstant.apply(StringConstant.java:63) at soot.toDex.StmtVisitor.caseAssignStmt(StmtVisitor.java:489) at soot.jimple.internal.JAssignStmt.apply(JAssignStmt.java:242) at soot.toDex.DexPrinter.toInstructions(DexPrinter.java:1511) at soot.toDex.DexPrinter.toMethodImplementation(DexPrinter.java:1174) at soot.toDex.DexPrinter.toMethods(DexPrinter.java:1083) at soot.toDex.DexPrinter.addAsClassDefItem(DexPrinter.java:656) at soot.toDex.DexPrinter.add(DexPrinter.java:1646) at soot.PackManager.writeClass(PackManager.java:1096) at soot.PackManager.lambda$writeOutput$1(PackManager.java:699) at soot.PackManager$$Lambda$2/1879083009.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Exception in thread "Thread-9" java.lang.RuntimeException: java.lang.OutOfMemoryError: GC overhead limit exceeded at soot.PackManager.writeOutput(PackManager.java:716) at soot.PackManager.writeDexOutput(PackManager.java:584) at soot.PackManager.writeOutput(PackManager.java:567) at soot.Main.run(Main.java:271) at soot.Main.main(Main.java:141) at org.ucr.ds.cd.utilities.Instrumenter.main(Instrumenter.java:407) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded at soot.toDex.ConstantVisitor.caseStringConstant(ConstantVisitor.java:86) at soot.jimple.StringConstant.apply(StringConstant.java:63) at soot.toDex.StmtVisitor.caseAssignStmt(StmtVisitor.java:489) at soot.jimple.internal.JAssignStmt.apply(JAssignStmt.java:242) at soot.toDex.DexPrinter.toInstructions(DexPrinter.java:1511) at soot.toDex.DexPrinter.toMethodImplementation(DexPrinter.java:1174) at soot.toDex.DexPrinter.toMethods(DexPrinter.java:1083) at soot.toDex.DexPrinter.addAsClassDefItem(DexPrinter.java:656) at soot.toDex.DexPrinter.add(DexPrinter.java:1646) at soot.PackManager.writeClass(PackManager.java:1096) at soot.PackManager.lambda$writeOutput$1(PackManager.java:699) at soot.PackManager$$Lambda$2/1879083009.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) java.lang.OutOfMemoryError: GC overhead limit exceeded at soot.toDex.ConstantVisitor.caseStringConstant(ConstantVisitor.java:86) at soot.jimple.StringConstant.apply(StringConstant.java:63) at soot.toDex.StmtVisitor.caseAssignStmt(StmtVisitor.java:489) at soot.jimple.internal.JAssignStmt.apply(JAssignStmt.java:242) at soot.toDex.DexPrinter.toInstructions(DexPrinter.java:1511) at soot.toDex.DexPrinter.toMethodImplementation(DexPrinter.java:1174) at soot.toDex.DexPrinter.toMethods(DexPrinter.java:1083) at soot.toDex.DexPrinter.addAsClassDefItem(DexPrinter.java:656) at soot.toDex.DexPrinter.add(DexPrinter.java:1646) at soot.PackManager.writeClass(PackManager.java:1096) at soot.PackManager.lambda$writeOutput$1(PackManager.java:699) at soot.PackManager$$Lambda$2/1879083009.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Ouuups... something went wrong! Sorry about that. Follow these steps to fix the problem: 1.) Are you sure you used the right command line? Click here to double-check: https://github.com/Sable/soot/wiki/Options-and-JavaDoc 2.) Not sure whether it's a bug? Feel free to discuss the issue on the Soot mailing list: https://github.com/Sable/soot/wiki/Getting-help 3.) Sure it's a bug? Click this link to report it. https://github.com/Sable/soot/issues/new?title=java.lang.RuntimeException+when+...&body=Steps+to+reproduce%3A%0A1.%29+...%0A%0AFiles+used+to+reproduce%3A+%0A...%0A%0ASoot+version%3A+%3Cpre%3Etrunk%3C%2Fpre%3E%0A%0ACommand+line%3A%0A%3Cpre%3E-w+-allow-phantom-refs+-process-multiple-dex+-force-android-jar+%2Fhome%2Fchungha%2FAndroid%2FSdk%2Fplatforms%2Fandroid-28%2Fandroid.jar+-src-prec+apk+-output-format+dex+-process-dir+%2Fhome%2Fchungha%2Fwork%2FAndroidSlicer%2Ftool%2FTwitter_v7.93.2-release.50_apkpure.com.apk%3C%2Fpre%3E%0A%0AMax+Memory%3A%0A%3Cpre%3E3775MB%3C%2Fpre%3E%0A%0AStack+trace%3A%0A%3Cpre%3Ejava.lang.RuntimeException%3A+java.lang.OutOfMemoryError%3A+GC+overhead+limit+exceeded%0A%09at+soot.PackManager.writeOutput%28PackManager.java%3A716%29%0A%09at+soot.PackManager.writeDexOutput%28PackManager.java%3A584%29%0A%09at+soot.PackManager.writeOutput%28PackManager.java%3A567%29%0A%09at+soot.Main.run%28Main.java%3A271%29%0A%09at+soot.Main.main%28Main.java%3A141%29%0A%09at+org.ucr.ds.cd.utilities.Instrumenter.main%28Instrumenter.java%3A407%29%0ACaused+by%3A+java.lang.OutOfMemoryError%3A+GC+overhead+limit+exceeded%0A%09at+soot.toDex.ConstantVisitor.caseStringConstant%28ConstantVisitor.java%3A86%29%0A%09at+soot.jimple.StringConstant.apply%28StringConstant.java%3A63%29%0A%09at+soot.toDex.StmtVisitor.caseAssignStmt%28StmtVisitor.java%3A489%29%0A%09at+soot.jimple.internal.JAssignStmt.apply%28JAssignStmt.java%3A242%29%0A%09at+soot.toDex.DexPrinter.toInstructions%28DexPrinter.java%3A1511%29%0A%09at+soot.toDex.DexPrinter.toMethodImplementation%28DexPrinter.java%3A1174%29%0A%09at+soot.toDex.DexPrinter.toMethods%28DexPrinter.java%3A1083%29%0A%09at+soot.toDex.DexPrinter.addAsClassDefItem%28DexPrinter.java%3A656%29%0A%09at+soot.toDex.DexPrinter.add%28DexPrinter.java%3A1646%29%0A%09at+soot.PackManager.writeClass%28PackManager.java%3A1096%29%0A%09at+soot.PackManager.lambda%24writeOutput%241%28PackManager.java%3A699%29%0A%09at+soot.PackManager%24%24Lambda%242%2F1879083009.run%28Unknown+Source%29%0A%09at+java.util.concurrent.ThreadPoolExecutor.runWorker%28ThreadPoolExecutor.java%3A1149%29%0A%09at+java.util.concurrent.ThreadPoolExecutor%24Worker.run%28ThreadPoolExecutor.java%3A624%29%0A%09at+java.lang.Thread.run%28Thread.java%3A748%29%0A%3C%2Fpre%3E Please be as precise as possible when giving us information on how to reproduce the problem. Thanks! Process finished with exit code 1
archer29m commented 5 years ago

For Twitter, this is the version that we used in the paper:

Twitter APK

This can be instrumented successfully. Generally since AndroidSlicer’s instrumenter is based on Soot, it inherits Soot’s static analysis size limitations, so that it cannot handle very large apps. Another hint to instrument large apps is to filter out the libraries that the analyzer knows that they are unrelated to his/her analysis, e.g., advertising, or authentication related libraries. For this purpose, you can uncomment line 173 of instrumenter.java to filter out the libraries that you don't want to instrument.

ChunghaSung commented 5 years ago

I tried with the app you gave, I still get the same error..

These are the arguments for the main class "Instrumenter", and I am using intellij.

com.twitter.android -w -allow-phantom-refs -process-multiple-dex -force-android-jar /home/chungha/Android/Sdk/platforms/android-28/android.jar -src-prec apk -output-format dex -process-dir /home/chungha/work/AndroidSlicer/tool/com.twitter.android.apk