After a ton of research, I figured out how to add support for debugging on old mono that targets net35. A Unity game may ship with depending on the Unity version:
It always ship with it if the version is older than 2017.1
It MAY ship with it if the version is between 2018.1 and 2019.1, but it is not the default because it's been deprecated by then (meaning the developer has to change the project setting or the setting was carried over from a Unity version upgrade of the project)
It CANNOT ship with it if the version is 2019.2 or newer (it was removed as it was already deprecated)
Adding this feature also fixes a regression from Doorstop 3 where it was no longer possible to debug using dnspy on these old mono because Doorstop 4 tried to enable the debugger and failed since it wasn't supported. Now, not only this works, but it no longer requires to have a patched mono.
To detect the mono, this was rather simple because checking the sources of unity's mono, I found old mono's runtime string can only start with v1 or v2 while new mono cannot and will typically start with v4. It just so happens that jit_init_version receives that string so I used it to detect which mono we are dealing with.
As for how this was done, old mono requires a different ordering of when to call jit_init_version where it must be called later than what you'll do on new mono. Since Doorstop was always doing what worked on new mono, it was crashing. After inspecting a development player of unity 5.5.4, I found that it would call debug_init right after jit_parse_options was called. What doorstop was doing is call jit_init_version right after debug_init, but it should have done it the other way around for old mono.
There was one last detail to handle: the protocol changed slightly such that to opt out of the suspend feature, it needs to have defer=y on top of the usual suspend=n. I simply changed the text it appends to accommodate depending on the mono used.
I did a bunch of tests with this using dnspy and 3 IDEs and everything seems to work on both old and new mono. That being said, I wanted to test the capabilities of this since I find this to be quite a huge deal so I did some more indepth tests and here's what I found:
DnSpy(ex): Debugging on old mono works as expected, but it no longer requires a patched mono. Suspend, attach or start (with or without suspend) works perfectly (the debugger doesn't need to be enabled when starting via dnspy, but it is required for attaching after the fact). On new mono, attaching after suspend seems to not work correctly, but given the results I got on other ides, it is likely a dnspy issue. Starting via dnspy on new mono still works with or without suspend. No symbols are needed: it just debug by decompilation
Rider: Has always worked with new mono perfectly using portable pdb from recompiled sources or even external source debugging without symbols (read more here: https://www.jetbrains.com/help/rider/Debugging_External_Code.html I only recently found out this was possible). What's new with this pr is it now supports old mono, but for usage with recompiled sources, the pdb format NEEDS to be either full pdb or mdb (obtained by invoking pdb2mdb on a full pdb), it cannot be a portable pdb. Additionally, if Doorstop is used with BepInEx, the preloader entrypoint MUST NOT be in UnityEngine.CoreModule because this breaks the evaluator (debugging and breakpoints still works though) This issue was known before this PR.
Visual Studio (with vstu): Always worked on new mono with portable pdb and recompiled sources, but it does not seem possible to debug externally via decompilation (it can detect the external assembly, but not auto decompile them even with the proper settings). On old mono, same as Rider except the symbols MUST be mdb so the csproj must be configured to generate full pdb and then pdb2mdb must be invoked on it.
Visual Studio Code (with unity extension): This is just broken. I can SOMEWHAT hit breakpoints with mdb symbols and recompiled sources, but it won't really update the UI and the unity extension spits errors. Meaning I can tell it can work, but it's just not wanting to work at the moment.
Due to the fact this pr also fixes a regression, I recommend that a release of doorstop is made with it (I can then send a pr to bepinex v5 and v6 so it gets the fix and the feature).
After a ton of research, I figured out how to add support for debugging on old mono that targets net35. A Unity game may ship with depending on the Unity version:
Adding this feature also fixes a regression from Doorstop 3 where it was no longer possible to debug using dnspy on these old mono because Doorstop 4 tried to enable the debugger and failed since it wasn't supported. Now, not only this works, but it no longer requires to have a patched mono.
To detect the mono, this was rather simple because checking the sources of unity's mono, I found old mono's runtime string can only start with v1 or v2 while new mono cannot and will typically start with v4. It just so happens that jit_init_version receives that string so I used it to detect which mono we are dealing with.
As for how this was done, old mono requires a different ordering of when to call jit_init_version where it must be called later than what you'll do on new mono. Since Doorstop was always doing what worked on new mono, it was crashing. After inspecting a development player of unity 5.5.4, I found that it would call debug_init right after jit_parse_options was called. What doorstop was doing is call jit_init_version right after debug_init, but it should have done it the other way around for old mono.
There was one last detail to handle: the protocol changed slightly such that to opt out of the suspend feature, it needs to have
defer=yon top of the usualsuspend=n. I simply changed the text it appends to accommodate depending on the mono used.I did a bunch of tests with this using dnspy and 3 IDEs and everything seems to work on both old and new mono. That being said, I wanted to test the capabilities of this since I find this to be quite a huge deal so I did some more indepth tests and here's what I found:
Due to the fact this pr also fixes a regression, I recommend that a release of doorstop is made with it (I can then send a pr to bepinex v5 and v6 so it gets the fix and the feature).