System.InsufficientExecutionStackException in 6.1.8 But it crashes/closes on 6.5.0

L-M022 commented 4 months ago

dnSpyEx version

6.1.8 - 6.5.0

Describe the Bug

I am trying to decompile a big method. I couldnt with Dnspy in both versions ( 6.1.8 - 6.5.0) but it was possible on ilSpy for some reason

How To Reproduce

If you decompile PlayZone Class from MuOrigin1 (private server version: https://www.muorigenlatino.com/index.html MuOrigenLatino9.0.apk), in version 6.1.8, it crashes and in 6.5.0 it closes automatically

Expected Behavior

It should be correctly decompiled like in ilSpy

Actual Behavior

In version 6.1.8, it crashes and in 6.5.0 it closes automatically

Additional Context

No response

ElektroKill commented 4 months ago

Hello, I downloaded the file you linked, opened Assembly-CSharp.dll, and found the PlayZone class in the root namespace. Decompilation of it yielded different results depending on the distribution which is quite interesting.

Using the latest build available here: https://github.com/dnSpyEx/dnSpy/actions/runs/8030324872

I got the following results:

.NET Framework distribution decompiles the class successfully. 64 bit version decompiles it rather quickly, 32 bit version takes some time but reaches the end
.NET 32 bit distribution - displays OutOfMemoryException when attempting to decompile. 32 bit version has 4GB ram limit. Interesting that .NET Framework version with the same limit works better.
.NET 64 bit distribution - Silently crashes without any displayed errors

I'm going to take a look at why the .NET 8 distributions perform so poorly considering its the newer runtime!

The reason this issue does not occur in modern ILSpy is that it uses a fundamentally different engine to that of dnSpy. dnSpy uses the old ILSpy 2.4 engine and ILSpy engine which is currently at version 8.

ElektroKill commented 4 months ago

Okay, there seems to be one VERY large method in this class, ProcessNetCommand which is the source of this issue. The .NET Framework version does not decompile this method instead emitting a body with an error message but decompiling the remaining methods.

The .NET versions do not handle this huge method very well causing "stronger" crashes. I will continue to investigate why that is!

The underlying reason is that the dnSpy decompiler cannot handle a method with other 70000 IL instructions in a single body with tons of branching resulting in failures. The root issue is the inefficiency of the decompiler.

dnSpyEx / dnSpy