virteman / smali

Automatically exported from code.google.com/p/smali
0 stars 0 forks source link

Deodexing android L preview framework files issue #225

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
I've extracted the dex files from oat and i'm trying to deodex them.
I'm encountering this issue with packet switch that prevents it to recompile 
successfully:

    .line 217
    .packed-switch 0x2bd0222
        +455938160
        +40435714
        +924766
        +1441792
        +196611
        -1779040253
        +21954681
        +24510464
        +1308295
        +1376264
        +1245448
        +476844123
        +1376264
        +646120020
        +582549876
        +151781396
        +270141458
        +565335
        +812515344
        +9578366
        +137955340
        +270663727
        +286923224
        +41314361
        +1113272
        +67964967
        +433590306
        +410521890
        -150994570
        +303693841
        +41192747
        +1177355
        +542380300
        +302523066
        +41156608
        +1177355
        +24383756
        +1177366
        +41292044
        +1113272
        +269291559
        +329784
        +797708402
        +270991368
        +581574740
        +4104
        +1048584
        +731783278
        +185335817
        -1293021071
        +51118088
        +798036082

Original issue reported on code.google.com by pipo...@gmail.com on 24 Oct 2014 at 2:48

GoogleCodeExporter commented 9 years ago
after further investigation, it seems the issue i'm having only on 
framework.jar is due to the fact it now uses classes.dex and classes2.dex.

Original comment by pipo...@gmail.com on 24 Oct 2014 at 4:00

GoogleCodeExporter commented 9 years ago
I'm not sure what you mean. The "odex" files for art are in a completely 
different format. However, they do contain the original dex file, which you 
should be able to extract and use as-is. As far as I know, they never contain 
what was previously known as an "odex" file, so they don't need to be 
"deodexed" - other than extracting the dex file.

Original comment by jesusfreke@jesusfreke.com on 7 Nov 2014 at 7:10

GoogleCodeExporter commented 9 years ago
I've of course extracted the original dex within the art file.
If it doesn't need to be deodex then i still have the above error after 
disassembling and reassembling it.

Also diffin the disassembled smali code from the dex within the art file and 
the disassembled one from the dex is different.
Also the first one contains many #disallowed odex opcode notes.

According to 
https://android.googlesource.com/platform/dalvik/+/android-4.4.4_r1/libdex/DexOp
codes.h and 
https://android.googlesource.com/platform/art/+/master/runtime/dex_instruction_l
ist.h it seems some opcodes are changed breaking compatibility with 
smali/backsmali.

According to anestib https://github.com/anestisb/oatdump_plus:

op  Dalvik  ART Comments
0x73    UNUSED  return-void-barrier Dalvik has return-void-barrier in 0xF1
0xE3    iget-volatile   iget-quick  Dalvik has iget-quick at 0xF3 -- *-volatile 
instructions are not included in new VM, fields volatile attr. are handled from 
MIR
0xE4    iput-volatile   iget-wide-quick Dalvik has iget-quick at 0xF3 -- *-volatile 
instructions are not included in new VM, fields volatile attr. are handled from 
MIR
0xE5    sget-volatile   iget-object-quick   Dalvik has iget-object-quick at 0xF4 -- 
*-volatile instructions are not included in new VM, fields volatile attr. are 
handled from MIR
0xE6    sput-volatile   iput-quick  Dalvik has iput-quick at 0xF5 -- *-volatile 
instructions are not included in new VM, fields volatile attr. are handled from 
MIR
0xE7    iget-object-volatile    iput-wide-quick Dalvik has iput-wide-quick at 0xF6 -- 
*-volatile instructions are not included in new VM, fields volatile attr. are 
handled from MIR
0xE8    iget-wide-volatile  iput-object-quick   Dalvik has iput-object quick at 0xF7 
-- *-volatile instructions are not included in new VM, fields volatile attr. 
are handled from MIR
0xE9    iput-wide-volatile  invoke-virtual-quick    Dalvik has invoke-virtual-quick at 
0xF8 -- *-volatile instructions are not included in new VM, fields volatile 
attr. are handled from MIR
0xEA    sget-wide-volatile  invoke-virtual/range-quick  Dalvik has 
invoke-virtual/range-quick at 0xF9 -- *-volatile instructions are not included 
in new VM, fields volatile attr. are handled from MIR
0xEB    sput-wide-volatile  UNUSED  *-volatile instructions are not included in new 
VM, fields volatile attr. are handled from MIR
0xEC    breakpoint  UNUSED  Instruction not included in new VM
0xED    throw-verification-error    UNUSED  Instruction not included in new VM
0xEE    execute-inline  UNUSED  Instruction not included in new VM
0xEF    execute-inline-range    UNUSED  Instruction not included in new VM
0xF0    invoke-object-init-range    UNUSED  Instruction not included in new VM
0xFA    invoke-super-quick  UNUSED  Instruction not included in new VM
0xFB    invoke-super-quick-range    UNUSED  Instruction not included in new VM
0xFC    iput-object-volatile    UNUSED  Instruction not included in new VM
0xFD    sget-object-volatile    UNUSED  Instruction not included in new VM
0xFE    sput-object-volatile    UNUSED  Instruction not included in new VM

Original comment by pipo...@gmail.com on 7 Nov 2014 at 5:45

GoogleCodeExporter commented 9 years ago
That switch instruction that you posted is nonsense. It's not even close to 
being a correct switch instruction. Without seeing the dex file that is causing 
that, I can't theorize what the problem might be.

Regarding the "disallowed odex opcodes" - if you're seeing that the dex file 
from an art ".odex" (oat) file has those, then yes, it will need to be deodexed.

Original comment by jesusfreke@jesusfreke.com on 7 Nov 2014 at 6:11

GoogleCodeExporter commented 9 years ago
Latinime dex is extracted from art, contains those disallowed odex opcodes so 
should be deodexed.
It decompile without errors but many files got those packed switch errors which 
prevent it to be recompile.

out\com\android\inputmethod\keyboard\internal\KeyboardTextsTable.smali[15513,22]
 Error for input '+': Invalid text

Calculator dex can't be decompiled at all giving 
java.lang.ArrayIndexOutOfBoundsException

Original comment by pipo...@gmail.com on 7 Nov 2014 at 6:38

GoogleCodeExporter commented 9 years ago
Aaaah, nevermind. I now see what you mean about the odex opcodes changing. I'll 
need to add support for the new odex opcodes.

And that is likely to be the cause of the original switch thing too.

Original comment by jesusfreke@jesusfreke.com on 7 Nov 2014 at 6:39

GoogleCodeExporter commented 9 years ago
Just provide my solution...
The startFieldOffset changed from 8 to 0 took me a lots of time...

Original comment by riddl...@gmail.com on 9 Nov 2014 at 4:23

Attachments:

GoogleCodeExporter commented 9 years ago
@riddl your solution doesn't seems to work.

I can confirm the issue is present in the final Lollipop release aswell.

wrong deodexed output sample:

    #disallowed odex opcode
    #iput-wide-volatile v0, v1, Landroid/system/OsConstants;->CAP_AUDIT_CONTROL:I
    nop

    move v0, v0

Correct deodexed output sample:

    invoke-virtual {v1}, Ldalvik/system/profiler/BinaryHprof$Tag;->ordinal()I

Original comment by pipo...@gmail.com on 14 Nov 2014 at 10:59

GoogleCodeExporter commented 9 years ago
I've also discovered that these new files are missing the dexopt(odex) header.
No MAGIC_36, they start with dex.035 not dey.036 so baksmali fail to even 
recognize them.

Original comment by pipo...@gmail.com on 14 Nov 2014 at 11:51

GoogleCodeExporter commented 9 years ago
Yeah, that was what made me think that the oat files only contained dex files - 
I didn't realize they were actually odex files, since they didn't have the dey 
header.

In any case, I've started work on better support for oat files. I've currently 
got some proof-of-concept code that will parse the oat file and find the dex 
files that it contains. Next step will be to clean that up, add support for the 
changed opcodes, and then figure out how I want baksmali to handle oat files 
with multiple dex files.

I'm currently thinking changing the output structure, so that each dex file in 
an oat file gets its own dir. I.e. for the boot.oat file, something like 
out/core, out/framework, etc.

Original comment by jesusfreke@jesusfreke.com on 14 Nov 2014 at 5:59

GoogleCodeExporter commented 9 years ago
@pipo
It needs to set options as below
baksmaliOptions options = new baksmaliOptions();
options.apiLevel = 21;
options.allowOdex = true;
options.deodex = true;

For some classes, it will need to extract all odex from boot.oat as classPath 
to resolve.
options.classPath = ...

Original comment by riddl...@gmail.com on 15 Nov 2014 at 12:45

GoogleCodeExporter commented 9 years ago
Share the jar.
java -jar bootoat2dex.jar boot.oat
It will output to 2 folders odex and dex(de-optimized)
I have tested pack these dex to original jar and push into device,
device can boot normally.

Original comment by riddl...@gmail.com on 15 Nov 2014 at 1:20

Attachments:

GoogleCodeExporter commented 9 years ago
[deleted comment]
GoogleCodeExporter commented 9 years ago
@riddl
your bootoat2dex doesn't work on released version of Lollipop (Samsung Galaxy 
S5 SM-G900F). Can you update utility or publish the code?

Original comment by saars...@gmail.com on 7 Dec 2014 at 12:17

GoogleCodeExporter commented 9 years ago
@saars
Samsung looks have a little different format.
I have added workaround to compatible (thanks svadev@xda provide sample file).
About Samsung's topic, there is some discussion in xda galaxy s5 forum.

Original comment by riddle_...@htc.com on 21 Dec 2014 at 4:51

Attachments:

GoogleCodeExporter commented 9 years ago
Just a quick update on this. The way the field offsets are calculated changed 
from dalvik to art. Unfortunately, it's not looking very feasible to emulator 
the exact behavior in java. The field ordering depends on a couple of 
implementation-specific behaviors in the exact c++ library being used. In 
particular, the exact implementation of std::sort, which is non-stable.. so the 
order of equal items is unspecified, and similarly, the exact implementation of 
priority_queue, which also does not guarantee the order of equal items.

So, I'm thinking it's going to be infeasible to update baksmali to be able to 
handle art odexes for now. Doing so would likely require some sort of on-device 
helper program to get the field offsets, similar to the original "deodexerant", 
and I'm not really interested in going that route.

Original comment by jesusfreke@jesusfreke.com on 28 Dec 2014 at 7:24

GoogleCodeExporter commented 9 years ago
There were some recent changes to art that make the field offsets more 
well-defined, and I do plan on adding support for deodexing odex files created 
by versions of art after these changes went in. However, note that there is no 
released build of Android that has all of those changes yet.

Original comment by jesusfreke@jesusfreke.com on 17 Mar 2015 at 3:42

GoogleCodeExporter commented 9 years ago
I'm encountering the same issue with sparse switch that prevents it to 
recompile successfully.

.sparse-switch
        0x3216137 -> +812580864
        0x4b1337 -> +34637299
        0x1000346 -> +196622
        0x61380369 -> +196611
        0x2852031b -> +1518206976
        0x10710001 -> +262272
        0x38748 -> +812580864
        0x369030c -> +34637300
        0x31b614b -> +196622
        0x1284e -> +196611
        -0x78b7ef8f -> +1519058944
        0x30c0003 -> +262272
        0x61490369 -> +812580864

Original comment by fred.and...@gmail.com on 24 Mar 2015 at 7:58

GoogleCodeExporter commented 9 years ago
Deodexing art odexes is not supported and does not work. There were some recent 
changes in art which will make it possible. I will add support in baksmali for 
deodexing odex files created by newer versions of art with these changes. But 
the changes aren't in any released version of Android yet.

The switch thing is likely just a symptom of some of the optimized opcodes 
changing between dalvik and art. That is simple enough to fix, but there are 
other problems with deodexing that I've mentioned previously.

Original comment by jesusfreke@jesusfreke.com on 24 Mar 2015 at 8:10

GoogleCodeExporter commented 9 years ago
@jesusfreke 
Thanks you very much, i'm glad to ear art odex will get supported for future 
android releases.

I'm not a programmer and not sure if this is related to the fields offsets 
problem you mentioned earlier but you may want to consider that riddl solution 
is working good on current art odex and he changed the startFieldOffset from 8 
to 0 and is not using any on-device helper program.

Original comment by pipo...@gmail.com on 27 Mar 2015 at 7:38

GoogleCodeExporter commented 9 years ago
You mean this: https://github.com/JesusFreke/smali/pull/30? That may work for 
specific cases, but it isn't guaranteed to work in general.

Original comment by jesusfreke@jesusfreke.com on 27 Mar 2015 at 8:28