[core] (1.5.0) APK fails to decompile with "Unsigned short value out of range: 71344"

bluemods commented 4 months ago

Issue details

The new release (1.5.0) fails to decompile APKs that previously decompiled many times without issue. Attached stack trace is below.

This error is caused by the smali tool attempting to repack a folder into a DEX file that has too many classes / methods in it (limit is 65536). This is probably caused by methods being inserted by JADX somehow into the smali then recompiling it, which then causes the per-dex limit to be exceeded. To get around this, you would split it into multiple dex files to make sure each one is under the limit. Not sure where I would look in the source to fix that though.

APK can be downloaded at: https://www.apkmirror.com/wp-content/themes/APKMirror/download.php?id=1237973&key=b005627028a9b01020c015bd1d083505135f1d1e

Relevant log output or stacktrace

ERROR: Smali compilation error:
com.android.tools.smali.util.ExceptionWithContext: Unsigned short value out of range: 71344
    at com.android.tools.smali.dexlib2.writer.DexDataWriter.writeUshort(DexDataWriter.java:115)
    at com.android.tools.smali.dexlib2.writer.DexWriter.writeMethods(DexWriter.java:542)
    at com.android.tools.smali.dexlib2.writer.DexWriter.writeTo(DexWriter.java:386)
    at com.android.tools.smali.dexlib2.writer.DexWriter.writeTo(DexWriter.java:370)
    at com.android.tools.smali.smali.Smali.assemble(Smali.java:136)
    at jadx.plugins.input.smali.SmaliConvert.lambda$compileSmali$1(SmaliConvert.java:60)
    at jadx.plugins.input.smali.SmaliConvert.collectSystemErrors(SmaliConvert.java:73)
    at jadx.plugins.input.smali.SmaliConvert.compileSmali(SmaliConvert.java:60)
    at jadx.plugins.input.smali.SmaliConvert.execute(SmaliConvert.java:38)
    at jadx.plugins.input.smali.SmaliInputPlugin.lambda$init$0(SmaliInputPlugin.java:22)
    at jadx.api.JadxDecompiler.loadInputFiles(JadxDecompiler.java:153)
    at jadx.api.JadxDecompiler.load(JadxDecompiler.java:119)
    at jadx.gui.JadxWrapper.open(JadxWrapper.java:72)
    at jadx.gui.ui.MainWindow.lambda$loadFiles$0(MainWindow.java:520)
    at jadx.core.utils.tasks.TaskExecutor.wrapTask(TaskExecutor.java:166)
    at jadx.core.utils.tasks.TaskExecutor.runStages(TaskExecutor.java:142)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1583)

ERROR: Smali compilation error:
com.android.tools.smali.util.ExceptionWithContext: Unsigned short value out of range: 71344
    at com.android.tools.smali.dexlib2.writer.DexDataWriter.writeUshort(DexDataWriter.java:115)
    at com.android.tools.smali.dexlib2.writer.DexWriter.writeMethods(DexWriter.java:542)
    at com.android.tools.smali.dexlib2.writer.DexWriter.writeTo(DexWriter.java:386)
    at com.android.tools.smali.dexlib2.writer.DexWriter.writeTo(DexWriter.java:370)
    at com.android.tools.smali.smali.Smali.assemble(Smali.java:136)
    at jadx.plugins.input.smali.SmaliConvert.lambda$compileSmali$1(SmaliConvert.java:60)
    at jadx.plugins.input.smali.SmaliConvert.collectSystemErrors(SmaliConvert.java:73)
    at jadx.plugins.input.smali.SmaliConvert.compileSmali(SmaliConvert.java:60)
    at jadx.plugins.input.smali.SmaliConvert.execute(SmaliConvert.java:38)
    at jadx.plugins.input.smali.SmaliInputPlugin.lambda$init$0(SmaliInputPlugin.java:22)
    at jadx.api.JadxDecompiler.loadInputFiles(JadxDecompiler.java:153)
    at jadx.api.JadxDecompiler.load(JadxDecompiler.java:119)
    at jadx.gui.JadxWrapper.open(JadxWrapper.java:72)
    at jadx.gui.ui.MainWindow.lambda$loadFiles$0(MainWindow.java:520)
    at jadx.core.utils.tasks.TaskExecutor.wrapTask(TaskExecutor.java:166)
    at jadx.core.utils.tasks.TaskExecutor.runStages(TaskExecutor.java:142)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1583)

Provide sample and class/method full name

No response

Jadx version

1.5.0

skylot commented 4 months ago

Hm, unfortunately smali doesn't support mutidex output, so it is hard to limit methods on input files. The only way is to do one to one compilation, and this will be very inefficient. Anyway, I will try to check smali sources for possible workaround.

@bluemods aside from smali error your claim that it works fine before looks doubful, because jadx doesn't support smali multidex in any version. Maybe you just load a whole folder containing apk or dex files and this folder also contains smali files now, so jadx fails to compile whese smali files. This is an only way I can think of to justify your claim :slightly_smiling_face:

jpstotz commented 4 months ago

May be a dumb question: Why should jadx trying to write a dex file when opening an APK file which already contains DEX files?

As far as I know SmaliConvert plugin is supposed to convert smali files to dex code. But the linked APK file does not contain smali files. So where do the smali files come from?

skylot commented 4 months ago

@jpstotz it is funny, but a lot of people confusing dex and smali and assume that jadx uses smali somehow during decompilation.

Correct statement is: jadx uses direct dex to source code translation and not using smali library during decompilation phase. Smali library used only to compile/convert input smali files to dex.

Another similar confusion I often saw, it is suggestion to convert dex to jar before using jadx, because people assume that jadx is a "java" decompiler :rofl:

skylot commented 4 months ago

I implemented compilation of one smali file at a time. I managed to avoid creation of temp files by using in memory store, also I used multithreaded compilation. As a result, new implementation can be faster than previous one :slightly_smiling_face:

bluemods commented 4 months ago

I built the master source myself with the latest commit and it decompiles fine now. Thank you

skylot / jadx