soot-oss / soot

Soot - A Java optimization framework
GNU Lesser General Public License v2.1
2.89k stars 711 forks source link

Annotation and dx #264

Closed algobardo closed 10 years ago

algobardo commented 10 years ago

You find the data to reproduce the problem on https://github.com/algobardo/sootReproduce

There you will find the original android framework class files.

If you try to run dx on any annotation class after you have loaded and written back with soot you will see

The detail of the dexopt error are the following: Invalid name : 'Ljava/lang/annotation/ElementType;'

raised in DexSwapVerify.cpp (dx tool) s = dexStringById(state->pDexFile, item->nameIdx); if (!dexIsValidMemberName(s)) { ALOGE("Invalid name: '%s'", s); return NULL; }

or Invalid name : 'SOURCE;' or Invalid name : 'FIELD'

does soot agrees with dx on the field descriptor format for annotations ?

ericbodden commented 10 years ago

Thanks for reporting this. Do you have any idea what could be wrong about 'Ljava/lang/annotation/ElementType;' and what the correct value should be? This certainly seems to work for Java. I wonder why dx has a problem with it.

algobardo commented 10 years ago

(My fault, the specific error in my previous message is coming from a dex --> dex instrumentation, but we can reproduce a similar problem more easily with class-->class and dx)

You can easily (hopefully) play with android classes here https://github.com/algobardo/sootReproduce

there you can instrument the entire framework with the right classpath, and have the resulting classes. Running dx --dex classfile is usually a good measure to establish if the generated class is compatible with dalvik

There is something different in the generated files (soot below, javac above) diff

that cause (in dx) UNEXPECTED TOP-LEVEL EXCEPTION: com.android.dx.cf.iface.ParseException: bad descriptor: METHOD at com.android.dx.cf.direct.AttributeListParser.parse(AttributeListParser.java:156) at com.android.dx.cf.direct.AttributeListParser.parseIfNecessary(AttributeListParser.java:115) at com.android.dx.cf.direct.AttributeListParser.getList(AttributeListParser.java:106) at com.android.dx.cf.direct.DirectClassFile.parse0(DirectClassFile.java:549) at com.android.dx.cf.direct.DirectClassFile.parse(DirectClassFile.java:406) at com.android.dx.cf.direct.DirectClassFile.parseToInterfacesIfNecessary(DirectClassFile.java:388) at com.android.dx.cf.direct.DirectClassFile.getMagic(DirectClassFile.java:251) at com.android.dx.command.dexer.Main.processClass(Main.java:665) at com.android.dx.command.dexer.Main.processFileBytes(Main.java:634) at com.android.dx.command.dexer.Main.access$600(Main.java:78) at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:572) at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:170) at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:144) at com.android.dx.command.dexer.Main.processOne(Main.java:596) at com.android.dx.command.dexer.Main.processAllFiles(Main.java:498) at com.android.dx.command.dexer.Main.runMonoDex(Main.java:264) at com.android.dx.command.dexer.Main.run(Main.java:230) at com.android.dx.command.dexer.Main.main(Main.java:199) at com.android.dx.command.Main.main(Main.java:103) Caused by: java.lang.IllegalArgumentException: bad descriptor: METHOD at com.android.dx.rop.type.Type.intern(Type.java:330) at com.android.dx.rop.cst.CstNat.getFieldType(CstNat.java:144) at com.android.dx.rop.cst.CstEnumRef.(CstEnumRef.java:36) at com.android.dx.cf.direct.AnnotationParser.parseValue(AnnotationParser.java:375) at com.android.dx.cf.direct.AnnotationParser.parseValue(AnnotationParser.java:399) at com.android.dx.cf.direct.AnnotationParser.parseElement(AnnotationParser.java:292) at com.android.dx.cf.direct.AnnotationParser.parseAnnotation(AnnotationParser.java:263) at com.android.dx.cf.direct.AnnotationParser.parseAnnotations(AnnotationParser.java:223) at com.android.dx.cf.direct.AnnotationParser.parseAnnotationAttribute(AnnotationParser.java:152) at com.android.dx.cf.direct.StdAttributeFactory.runtimeVisibleAnnotations(StdAttributeFactory.java:632) at com.android.dx.cf.direct.StdAttributeFactory.parse0(StdAttributeFactory.java:97) at com.android.dx.cf.direct.AttributeFactory.parse(AttributeFactory.java:96) at com.android.dx.cf.direct.AttributeListParser.parse(AttributeListParser.java:141) ... 18 more ...while parsing attributes[1] ...while parsing android/test/FlakyTest.class

the actual code, that we reach with param "METHOD" is below

/**
     * Returns the unique instance corresponding to the type with the
     * given descriptor. See vmspec-2 sec4.3.2 for details on the
     * field descriptor syntax. This method does <i>not</i> allow
     * {@code "V"} (that is, type {@code void}) as a valid
     * descriptor.
     *
     * @param descriptor {@code non-null;} the descriptor
     * @return {@code non-null;} the corresponding instance
     * @throws IllegalArgumentException thrown if the descriptor has
     * invalid syntax
     */
    public static Type intern(String descriptor) {
        Type result;
        synchronized (internTable) {
            result = internTable.get(descriptor);
        }
        if (result != null) {
            return result;
        }

        char firstChar;
        try {
            firstChar = descriptor.charAt(0);
        } catch (IndexOutOfBoundsException ex) {
            // Translate the exception.
            throw new IllegalArgumentException("descriptor is empty");
        } catch (NullPointerException ex) {
            // Elucidate the exception.
            throw new NullPointerException("descriptor == null");
        }

        if (firstChar == '[') {
            /*
             * Recursively strip away array markers to get at the underlying
             * type, and build back on to form the result.
             */
            result = intern(descriptor.substring(1));
            return result.getArrayType();
        }

        /*
         * If the first character isn't '[' and it wasn't found in the
         * intern cache, then it had better be the descriptor for a class.
         */

        int length = descriptor.length();
        if ((firstChar != 'L') ||
            (descriptor.charAt(length - 1) != ';')) {
            throw new IllegalArgumentException("bad descriptor: " + descriptor);
        }

        /*
         * Validate the characters of the class name itself. Note that
         * vmspec-2 does not have a coherent definition for valid
         * internal-form class names, and the definition here is fairly
         * liberal: A name is considered valid as long as it doesn't
         * contain any of '[' ';' '.' '(' ')', and it has no more than one
         * '/' in a row, and no '/' at either end.
         */

        int limit = (length - 1); // Skip the final ';'.
        for (int i = 1; i < limit; i++) {
            char c = descriptor.charAt(i);
            switch (c) {
                case '[':
                case ';':
                case '.':
                case '(':
                case ')': {
                    throw new IllegalArgumentException("bad descriptor: " + descriptor);
                }
                case '/': {
                    if ((i == 1) ||
                        (i == (length - 1)) ||
                        (descriptor.charAt(i - 1) == '/')) {
                        throw new IllegalArgumentException("bad descriptor: " + descriptor);
                    }
                    break;
                }
            }
        }

        result = new Type(descriptor, BT_OBJECT);
        return putIntern(result);
    }
ericbodden commented 10 years ago

I am sorry but why is the sootReproduce so huge? Can you please produce a small, minimal test case that we ca work with? I am not going to download a gigabyte of stuff onto my computer just to debug this. Thanks

algobardo commented 10 years ago

I removed the android-platforms (230Mb)....but the classpath and soot is still needed. I also removed the framework classes, so now you will have lots of phantom refs. https://github.com/algobardo/smallerSootReproduce

type gradle run to instrument the classes in res.

then cd res dx --dex android..... (with a name of one of the instrumented classes)

You will see the error returned by dx