search

Project-ARTist / dexterous

Library and standalone CLI tool for apk/dex merging, repackaging and signing. Can also get used as a dex analyzer framework.
Other
48 stars 16 forks source link

Cannot merge codelib into systemserver #7

Closed schrnz closed 6 years ago

schrnz commented 6 years ago

When I try to use the current dexterous version @ fd5a845390585e0341ded2bf97a44dcbb68ee4a5 to merge a codelib into the systemserver (services.jar), the following error occurs:

Partially merging CodeLib into services.jar using dexterous
I/Dexterous: Loading DexFile: services.apk:classes.dex
I/Dexterous: Loading DexFile: services.apk:classes2.dex
I/Dexterous: DONE Loading 01 Dexfiles (services.apk)
I/Dexterous: Loading DexFile: app-debug.apk:classes.dex
I/Dexterous: Loading DexFile: app-debug.apk:classes2.dex
I/Dexterous: DONE Loading 01 Dexfiles (app-debug.apk)
I/Dexterously: # MULTIDEX File. DexFile Count: 2
I/Dexterously: 
I/Dexterously: # DEXFILE: services.apk:classes.dex #############################################
I/Dexterously: 
I/Loggy: ----------------------------------------------------------------------------
I/Loggy: Section |TYPE:                       null | OFF:       -1 | SIZE:        0 |
I/Loggy: Section |TYPE:                       null | OFF:       -1 | SIZE:        0 |
I/Loggy: Section |TYPE:                     HEADER | OFF:        0 | SIZE:        1 |
I/Loggy: Section |TYPE:                 STRING_IDS | OFF:      112 | SIZE:    63478 |
I/Loggy: Section |TYPE:                   TYPE_IDS | OFF:   254024 | SIZE:     4637 |
I/Loggy: Section |TYPE:                  PROTO_IDS | OFF:   272572 | SIZE:     9860 |
I/Loggy: Section |TYPE:                  FIELD_IDS | OFF:   390892 | SIZE:    19115 |
I/Loggy: Section |TYPE:                 METHOD_IDS | OFF:   543812 | SIZE:    34535 |
I/Loggy: Section |TYPE:                 CLASS_DEFS | OFF:   820092 | SIZE:     2616 |
I/Loggy: Section |TYPE:   ANNOTATION_SET_REF_LISTS | OFF:   903804 | SIZE:      352 |
I/Loggy: Section |TYPE:            ANNOTATION_SETS | OFF:   908676 | SIZE:     3517 |
I/Loggy: Section |TYPE:                 CODE_ITEMS | OFF:   945024 | SIZE:    24583 |
I/Loggy: Section |TYPE: ANNOTATION_DIRECTORY_ITEMS | OFF:  4128108 | SIZE:     2239 |
I/Loggy: Section |TYPE:                 TYPE_LISTS | OFF:  4193532 | SIZE:     6329 |
I/Loggy: Section |TYPE:          STRING_DATA_ITEMS | OFF:  4262876 | SIZE:    63478 |
I/Loggy: Section |TYPE:            DEBUG_INFO_ITEM | OFF:  5763692 | SIZE:    22550 |
I/Loggy: Section |TYPE:            ANNOTATION_ITEM | OFF:  6587462 | SIZE:     3781 |
I/Loggy: Section |TYPE:        ENCODED_ARRAY_ITEMS | OFF:  6656830 | SIZE:      539 |
I/Loggy: Section |TYPE:           CLASS_DATA_ITEMS | OFF:  6670105 | SIZE:     2604 |
I/Loggy: Section |TYPE:                   MAP_LIST | OFF:  6880792 | SIZE:        1 |
I/Loggy: ----------------------------------------------------------------------------
I/Dexterously: 
I/Dexterously: # DEXFILE: app-debug.apk:classes.dex #############################################
I/Dexterously: 
I/Loggy: ----------------------------------------------------------------------------
I/Loggy: Section |TYPE:                       null | OFF:       -1 | SIZE:        0 |
I/Loggy: Section |TYPE:                       null | OFF:       -1 | SIZE:        0 |
I/Loggy: Section |TYPE:                     HEADER | OFF:        0 | SIZE:        1 |
I/Loggy: Section |TYPE:                 STRING_IDS | OFF:      112 | SIZE:    21234 |
I/Loggy: Section |TYPE:                   TYPE_IDS | OFF:    85048 | SIZE:     2352 |
I/Loggy: Section |TYPE:                  PROTO_IDS | OFF:    94456 | SIZE:     3489 |
I/Loggy: Section |TYPE:                  FIELD_IDS | OFF:   136324 | SIZE:    10394 |
I/Loggy: Section |TYPE:                 METHOD_IDS | OFF:   219476 | SIZE:    17438 |
I/Loggy: Section |TYPE:                 CLASS_DEFS | OFF:   358980 | SIZE:     1626 |
I/Loggy: Section |TYPE:                 CODE_ITEMS | OFF:   411012 | SIZE:    11730 |
I/Loggy: Section |TYPE:            DEBUG_INFO_ITEM | OFF:  1237269 | SIZE:    10561 |
I/Loggy: Section |TYPE:                 TYPE_LISTS | OFF:  1463400 | SIZE:     2079 |
I/Loggy: Section |TYPE:          STRING_DATA_ITEMS | OFF:  1484594 | SIZE:    21234 |
I/Loggy: Section |TYPE:            ANNOTATION_ITEM | OFF:  2021194 | SIZE:     2040 |
I/Loggy: Section |TYPE:           CLASS_DATA_ITEMS | OFF:  2054433 | SIZE:     1516 |
I/Loggy: Section |TYPE:        ENCODED_ARRAY_ITEMS | OFF:  2151369 | SIZE:      291 |
I/Loggy: Section |TYPE:            ANNOTATION_SETS | OFF:  2167192 | SIZE:     1916 |
I/Loggy: Section |TYPE:   ANNOTATION_SET_REF_LISTS | OFF:  2189004 | SIZE:      144 |
I/Loggy: Section |TYPE: ANNOTATION_DIRECTORY_ITEMS | OFF:  2191632 | SIZE:     1436 |
I/Loggy: Section |TYPE:                   MAP_LIST | OFF:  2237768 | SIZE:        1 |
I/Loggy: ----------------------------------------------------------------------------
I/MethodFilter: Annotationtype used for whitelisting:Lsaarland/cispa/artist/codelib/CodeLib$Inject;
Exception in thread "main" java.lang.IllegalArgumentException: position=88867372 length=2237988
    at comm.android.dex.Dex.open(Dex.java:191)
    at comm.android.dx.merge.MethodFilter.getString(MethodFilter.java:136)
    at comm.android.dx.merge.MethodFilter.getMethodString(MethodFilter.java:150)
    at comm.android.dx.merge.MethodFilter.initializeWhitelist(MethodFilter.java:230)
    at comm.android.dx.merge.MethodFilter.<init>(MethodFilter.java:387)
    at comm.android.dex.Dex.setWhitelistedAnnotation(Dex.java:358)
    at saarland.cispa.dexterous.cli.Dexterously.mergeCodeLib(Dexterously.java:617)
    at saarland.cispa.dexterous.cli.Main.main(Main.java:49)

It does NOT occur when I use the dexterous version from before the annotation-based whitelisting patch (tested: 3253508aed834374c2007c49a349a233e9ae03ae).

@alfink do you have an intuition what is going wrong here?

alfink commented 6 years ago

Seems to be the bug I fixed in #5

schrnz commented 6 years ago

Using #5 it's a MergeException now: 

I/Dexterous: Loading DexFile: services.apk:classes.dex
I/Dexterous: Loading DexFile: services.apk:classes2.dex
I/Dexterous: DONE Loading 01 Dexfiles (services.apk)
I/Dexterous: Loading DexFile: CodeLib.apk:classes.dex
I/Dexterous: Loading DexFile: CodeLib.apk:classes2.dex
I/Dexterous: DONE Loading 01 Dexfiles (CodeLib.apk)
I/Dexterously: # MULTIDEX File. DexFile Count: 2
I/Dexterously: 
I/Dexterously: # DEXFILE: services.apk:classes.dex #############################################
I/Dexterously: 
I/Loggy: ----------------------------------------------------------------------------
I/Loggy: Section |TYPE:                       null | OFF:       -1 | SIZE:        0 |
I/Loggy: Section |TYPE:                       null | OFF:       -1 | SIZE:        0 |
I/Loggy: Section |TYPE:                     HEADER | OFF:        0 | SIZE:        1 |
I/Loggy: Section |TYPE:                 STRING_IDS | OFF:      112 | SIZE:    63478 |
I/Loggy: Section |TYPE:                   TYPE_IDS | OFF:   254024 | SIZE:     4637 |
I/Loggy: Section |TYPE:                  PROTO_IDS | OFF:   272572 | SIZE:     9860 |
I/Loggy: Section |TYPE:                  FIELD_IDS | OFF:   390892 | SIZE:    19115 |
I/Loggy: Section |TYPE:                 METHOD_IDS | OFF:   543812 | SIZE:    34535 |
I/Loggy: Section |TYPE:                 CLASS_DEFS | OFF:   820092 | SIZE:     2616 |
I/Loggy: Section |TYPE:   ANNOTATION_SET_REF_LISTS | OFF:   903804 | SIZE:      352 |
I/Loggy: Section |TYPE:            ANNOTATION_SETS | OFF:   908676 | SIZE:     3517 |
I/Loggy: Section |TYPE:                 CODE_ITEMS | OFF:   945024 | SIZE:    24583 |
I/Loggy: Section |TYPE: ANNOTATION_DIRECTORY_ITEMS | OFF:  4128108 | SIZE:     2239 |
I/Loggy: Section |TYPE:                 TYPE_LISTS | OFF:  4193532 | SIZE:     6329 |
I/Loggy: Section |TYPE:          STRING_DATA_ITEMS | OFF:  4262876 | SIZE:    63478 |
I/Loggy: Section |TYPE:            DEBUG_INFO_ITEM | OFF:  5763692 | SIZE:    22550 |
I/Loggy: Section |TYPE:            ANNOTATION_ITEM | OFF:  6587462 | SIZE:     3781 |
I/Loggy: Section |TYPE:        ENCODED_ARRAY_ITEMS | OFF:  6656830 | SIZE:      539 |
I/Loggy: Section |TYPE:           CLASS_DATA_ITEMS | OFF:  6670105 | SIZE:     2604 |
I/Loggy: Section |TYPE:                   MAP_LIST | OFF:  6880792 | SIZE:        1 |
I/Loggy: ----------------------------------------------------------------------------
I/Dexterously: 
I/Dexterously: # DEXFILE: CodeLib.apk:classes.dex #############################################
I/Dexterously: 
I/Loggy: ----------------------------------------------------------------------------
I/Loggy: Section |TYPE:                       null | OFF:       -1 | SIZE:        0 |
I/Loggy: Section |TYPE:                       null | OFF:       -1 | SIZE:        0 |
I/Loggy: Section |TYPE:                     HEADER | OFF:        0 | SIZE:        1 |
I/Loggy: Section |TYPE:                 STRING_IDS | OFF:      112 | SIZE:      927 |
I/Loggy: Section |TYPE:                   TYPE_IDS | OFF:     3820 | SIZE:      192 |
I/Loggy: Section |TYPE:                  PROTO_IDS | OFF:     4588 | SIZE:      195 |
I/Loggy: Section |TYPE:                  FIELD_IDS | OFF:     6928 | SIZE:      198 |
I/Loggy: Section |TYPE:                 METHOD_IDS | OFF:     8512 | SIZE:      611 |
I/Loggy: Section |TYPE:                 CLASS_DEFS | OFF:    13400 | SIZE:       92 |
I/Loggy: Section |TYPE:   ANNOTATION_SET_REF_LISTS | OFF:    16344 | SIZE:       13 |
I/Loggy: Section |TYPE:            ANNOTATION_SETS | OFF:    16452 | SIZE:      159 |
I/Loggy: Section |TYPE:                 CODE_ITEMS | OFF:    18112 | SIZE:      432 |
I/Loggy: Section |TYPE: ANNOTATION_DIRECTORY_ITEMS | OFF:    41232 | SIZE:       88 |
I/Loggy: Section |TYPE:                 TYPE_LISTS | OFF:    43544 | SIZE:       83 |
I/Loggy: Section |TYPE:          STRING_DATA_ITEMS | OFF:    44260 | SIZE:      927 |
I/Loggy: Section |TYPE:            DEBUG_INFO_ITEM | OFF:    63692 | SIZE:      430 |
I/Loggy: Section |TYPE:            ANNOTATION_ITEM | OFF:    69502 | SIZE:      207 |
I/Loggy: Section |TYPE:        ENCODED_ARRAY_ITEMS | OFF:    72069 | SIZE:        4 |
I/Loggy: Section |TYPE:           CLASS_DATA_ITEMS | OFF:    72095 | SIZE:       89 |
I/Loggy: Section |TYPE:                   MAP_LIST | OFF:    75532 | SIZE:        1 |
I/Loggy: ----------------------------------------------------------------------------
I/MethodFilter: Annotationtype used for whitelisting:Lsaarland/cispa/artist/codelib/CodeLib$Inject;
comm.android.dx.merge.DexMerger$MergeException
    at comm.android.dx.merge.MethodFilter.initializeWhitelist(MethodFilter.java:171)
    at comm.android.dx.merge.MethodFilter.<init>(MethodFilter.java:387)
    at comm.android.dex.Dex.setWhitelistedAnnotation(Dex.java:358)
    at saarland.cispa.dexterous.cli.Dexterously.mergeCodeLib(Dexterously.java:617)
    at saarland.cispa.dexterous.cli.Main.main(Main.java:49)
alfink commented 6 years ago

The definition of the Annotation Lsaarland/cispa/artist/codelib/CodeLib$Inject seems to be missing in your Codelib. Somehow the InvalidArgumentException wrapped in the MergeException was not printed.

schrnz commented 6 years ago

I see, it was an error in one of our build scripts that used the wrong codelib version, thanks!

Anyway, about the MergeException: Is it expected that the underlying exception is printed/explained? Would help for debugging

alfink commented 6 years ago

It is expected that the underlying exception is printed. This is the behavior in ARTist GUI but missing in the CLI. I will add this to #5.