search

google / smali

Other
212 stars 35 forks source link

assert failed in 3.0.7 #57

Closed eirv closed 1 month ago

eirv commented 6 months ago

I upgraded my own project's dexlib2 dependency to 3.0.7, and I tried to use dexlib2 to modify a dex built by d8, but it reported an error. I found that the problem was caused by this commit

java.lang.AssertionError
    at com.android.tools.smali.util.ArraySortedSet.assertSorted(ArraySortedSet.java:78)
    at com.android.tools.smali.util.ArraySortedSet.<init>(ArraySortedSet.java:59)
    at com.android.tools.smali.util.ArraySortedSet.of(ArraySortedSet.java:67)
    at com.android.tools.smali.dexlib2.writer.pool.PoolClassDef.<init>(PoolClassDef.java:75)
    at com.android.tools.smali.dexlib2.writer.pool.ClassPool.intern(ClassPool.java:98)
    at com.android.tools.smali.dexlib2.writer.pool.DexPool.internClass(DexPool.java:129)
    at xxx
melcz commented 6 months ago

An assertion was added to ensure that the creation of the structure was done with a sorted array, I am unfamiliar with an usage path that would break this assumption. Could you please provide a smaller sample of reproducibility?

eirv commented 6 months ago

An assertion was added to ensure that the creation of the structure was done with a sorted array, I am unfamiliar with an usage path that would break this assumption. Could you please provide a smaller sample of reproducibility?

OK. It is here

I tried to modify the dex using dexlib2 on my PC (openjdk21) and although it didn't crash, the modified dex fails to load, and modifying it on android crashes.

melcz commented 4 months ago

Disassembling the generated dex files works fine with:

java -jar smali/baksmali/build/libs/baksmali-3.0.7-5f8b0491-dirty-fat.jar dis DexlibTest/app/build/outputs/apk/release/app-release-unsigned.apk

Could you detail what modifications you tried that caused the failure?

eirv commented 4 months ago

Disassembling the generated dex files works fine with:

java -jar smali/baksmali/build/libs/baksmali-3.0.7-5f8b0491-dirty-fat.jar dis DexlibTest/app/build/outputs/apk/release/app-release-unsigned.apk

Could you detail what modifications you tried that caused the failure?

I'm not talking about smali/baksmali, the bug occurs after modifying the class name using dexlib2, you can try running that demo and you can see that it crashes. There is no problem modifying the class name using dexlib2 in version 3.0.3. You can modify the dex normally and it can be loaded by Android as well

melcz commented 4 months ago

Got it, this looks like it could be solved in the constructor for PoolClassDef. I'll try to write a test to reproduce and submit it with a fix. Thanks for the sample app!

melcz commented 4 months ago

It doesn't seem like the issue is consistently reproducible. I will still do the change for PoolClassDef, but this behavior could be reverted in the future.