soot-oss / soot

Soot - A Java optimization framework
GNU Lesser General Public License v2.1
2.86k stars 705 forks source link

Instrumented Android app cannot be installed! #683

Closed MiJack closed 6 years ago

MiJack commented 7 years ago

Hi,guys! I met some problem about instrumented Android app.After instrumenting Android app using Soot and signing it, I got an app named app-after-soot.apk, but got the error named INSTALL_PARSE_FAILED_UNEXPECTED_EXCEPTION using adb to install it.However , after decompiling and repackaging the apk app-after-soot.apk using ApkTool and signing the output apk ,I can install the output apk. So ,does Soot have some bugs on the dex output format or my code have some errors? Thanks!

MiJack commented 7 years ago

The code is below:

package com.mijack.sootdemo;

import soot.*;
import soot.jimple.*;
import soot.options.Options;

import java.util.Collections;
import java.util.Iterator;
import java.util.Map;

public class Main {
    public static final String APK = ...;
    public static final String ANDROID_JAR = ...;

    public static void main(String[] args) {
        init();

        Scene.v().addBasicClass("java.io.PrintStream", SootClass.SIGNATURES);
        Scene.v().addBasicClass("java.lang.System", SootClass.SIGNATURES);

        PackManager.v().getPack("jtp").add(new Transform("jtp.demo", new BodyTransformer() {
            @Override
            protected void internalTransform(final Body b, String s, Map<String, String> map) {
                final PatchingChain<Unit> units = b.getUnits();
                System.out.println(b.getMethod().getSignature());
                //important to use snapshotIterator here
                for (Iterator<Unit> iter = units.snapshotIterator(); iter.hasNext(); ) {
                    final Unit u = iter.next();
                    u.apply(/*new PrintStmtSwitch() {}*/ new AbstractStmtSwitch() {
                        @Override
                        public void caseIfStmt(IfStmt stmt) {
                            System.out.println("find a if statement");
                            Local tmpRef = addTmpRef(b);
                            Local tmpString = addTmpString(b);
                            // insert "tmpRef = java.lang.System.out;"
                            units.insertBefore(Jimple.v().newAssignStmt(
                                    tmpRef, Jimple.v().newStaticFieldRef(
                                            Scene.v().getField("<java.lang.System: java.io.PrintStream out>").makeRef())), u);

                            // insert "tmpLong = 'HELLO';"
                            units.insertBefore(Jimple.v().newAssignStmt(tmpString,
                                    StringConstant.v("GotoStmt")), u);

                            // insert "tmpRef.println(tmpString);"
                            SootMethod toCall = Scene.v().getSootClass("java.io.PrintStream").getMethod("void println(java.lang.String)");
                            units.insertBefore(Jimple.v().newInvokeStmt(
                                    Jimple.v().newVirtualInvokeExpr(tmpRef, toCall.makeRef(), tmpString)), u);

                            //check that we did not mess up the Jimple
                            b.validate();

                        }

                    } );
                }
            }
        }));

        PackManager.v().runPacks();
        PackManager.v().writeOutput();
    }

    private static Local addTmpRef(Body body) {
        Local tmpRef = Jimple.v().newLocal("tmpRef", RefType.v("java.io.PrintStream"));
        body.getLocals().add(tmpRef);
        return tmpRef;
    }

    private static Local addTmpString(Body body) {
        Local tmpString = Jimple.v().newLocal("tmpString", RefType.v("java.lang.String"));
        body.getLocals().add(tmpString);
        return tmpString;
    }
    private static void init() {
        Options.v().set_allow_phantom_refs(true);
        Options.v().set_prepend_classpath(true);
        Options.v().set_validate(true);
        Options.v().set_whole_program(true);
        Options.v().set_force_overwrite(true);
        Options.v().set_output_format(Options.output_format_dex);
        Options.v().set_process_dir(Collections.singletonList(APK));
        Options.v().set_android_jars(ANDROID_JAR);
        Options.v().set_src_prec(Options.src_prec_apk);
        Options.v().set_soot_classpath(ANDROID_JAR);

        Scene.v().loadNecessaryClasses();
    }
}
lorinbeer commented 7 years ago

Hi @MiJack and the Soot team.

I have run into the same issue. Running soot to instrument an android app fails if --allow-phantom-refs is not used.

Using --allow-phantom-refs builds the apk, which can then be signed and zipaligned.

But attempting to install the apk results in Failed to install sootOutput/app-debug.apk: Failure [INSTALL_PARSE_FAILED_UNEXPECTED_EXCEPTION: Failed reading classes.dex in android.util.jar.StrictJarFile@597c07a: META-INF/MANIFEST.MF has invalid digest for classes.dex in classes.dex]

Now, my suspicion is that the culprit is the android.jar I have soot pointed to. The repo containing full android jars appears to be about 4 years out of date. I've pulled the android.jar, framework.jar and others from an emulated device, but nothing tried has worked so far.

My last idea was to compile the full android source and see if that will work.

Suggestions are welcome.

wishma2013 commented 6 years ago

I'm trying recombine a classes.dex file and replace with the one in an apk file.After all the step and final install the apk via adb.I got a similar error:adb: failed to install app-debug.apk: Failure [INSTALL_PARSE_FAILED_UNEXPECTED_EXCEPTION: Failed reading classes.dex in android.util.jar.StrictJarFile@1c3374b: META-INF/MANIFEST.MF has invalid digest for classes.dex in classes.dex]. I'm trying to resolve it.

ericbodden commented 6 years ago

Hello. APKs are signed. You need to recreate a valid signature.

Best wishes Eric

On 19. Jun 2018, at 19:24, wishma2013 notifications@github.com wrote:

I'm trying recombine a classes.dex file and replace with the one in an apk file.After all the step and final install the apk via adb.I got a similar error:adb: failed to install app-debug.apk: Failure [INSTALL_PARSE_FAILED_UNEXPECTED_EXCEPTION: Failed reading classes.dex in android.util.jar.StrictJarFile@1c3374b: META-INF/MANIFEST.MF has invalid digest for classes.dex in classes.dex]. I'm trying to resolve it.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

codeMinter commented 6 years ago

Are we saying if we sign post instrumentation using self-signed cert, that would also throw this error as APK's were already signed ? My scenario is as below:

  1. Download APK
  2. Do Instumentation
  3. Sign using Jarsigner with self-signed cert
  4. zipalign and install
mbenz89 commented 6 years ago

No, signing with a self-signed certificate should work just fine. However, you have to zipalign before you sign the APK since you'll invalidate the calculated signature otherwise.

codeMinter commented 6 years ago

Yup we were already doing it. But then Android documentation said if you use jarsigner to sign, sign first and then do zipalign - so we tried both variants, but we always get same error when installing updated apk...

$ adb install target-aligned.apk 
adb: failed to install target-aligned.apk: Failure [INSTALL_PARSE_FAILED_UNEXPECTED_EXCEPTION: Failed reading classes.dex in android.util.jar.StrictJarFile@a7397fe: META-INF/MANIFEST.MF has invalid digest for classes.dex in classes.dex]
wishma2013 commented 6 years ago

Thank you @ericbodden .I have deleted MANIFEST.MF/CERT.SF/CERT.RSA,and re-sign the apk.The error has gone.

codeMinter commented 6 years ago

Are we saying one has to manually do this step of removing CERT? I suggest we make it configurable in soot so everyone doesn't needs to write this but just flag soot that we want to have CERT moved out. Can we re-open and add this in soot so users just flag soot via Options setter ?

mbenz89 commented 6 years ago

Soot should already do that by default in DexPrinter.

Maybe you can track down why that does not work for you.