Support for Roslyn inside wine-mono

[Onyx47]

Recently I found this issue opened on Space Engineers GitHub: https://github.com/KeenSoftwareHouse/SpaceEngineers/pull/610

I can confirm that patching the game's IL to implement the requested change does enable it to start up and run successfully. Unfortunately, what still doesn't work is mods or scripts that can be ran in-game. Both are written in C# and from what I've been told it uses Roslyn to compile the code at startup. Loading the game with mods does not produce a crash but all compilation fails. Here is the full Proton log from starting the game, trying to load a world with mods and closing the game:

Proton log

This seems to be the relevant part:

[000000000000011c:] EXCEPTION handling: System.DllNotFoundException: Microsoft.DiaSymReader.Native.amd64.dll assembly:<unknown assembly> type:<unknown type> member:(null)
419229.856:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 108
419229.856:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 109
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 110
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 111
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 112
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 113
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 114
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 115
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 116
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 117
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 118
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 119
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 120
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 121
[000000000000011c:] EXCEPTION handling: System.PlatformNotSupportedException: Operation is not supported on this platform.
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 122
419229.857:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 123
419229.859:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 124
[000000000000011c:] EXCEPTION handling: System.DllNotFoundException: Operation is not supported on this platform.
419229.859:0118:011c:trace:seh:RtlGrowFunctionTable 000000009CF88ED0, 125
[000000000000011c:] EXCEPTION handling: Microsoft.DiaSymReader.SymUnmanagedWriterException: Operation is not supported on this platform.

Is it possible to build wine-mono with required assemblies, even if it's only a custom build if this is outside of scope for the project?

[madewokherd]

FWIW, I ended up fixing the Queue issue in wine-mono. It just hasn't been in a release yet. So a CI build from https://github.com/madewokherd/wine-mono/actions may remove the need to patch the IL.

My immediate question is: where is this dll supposed to come from? On my Windows machine, I don't see it in Microsoft.NET, so I don't think it's shipped as part of .NET Framework (which doesn't necessarily mean it's out-of-scope - both monoDX and FNA are replacing components that aren't technically part of .NET Framework but are from Microsoft and commonly installed via redists).

So my guess would be that either the dll is shipped with the game and isn't being found for some reason, or Mono needs to load it in a situation where .NET Framework does not (a known example of this is that on .NET Framework a class field can have a type that can't be loaded, as long as the field itself is unused, in that case we'd probably add a work-around to Proton). It could also be a case where the requirement is because of a code-path the game isn't supposed to take.

[madewokherd]

The game is kind of intimidating to me, so detailed steps to reproduce the bug using a specific mod would help.

[Onyx47]

In my prefix with dotnet installed they seem to be installed as part of dotnet:

$ ~/Games/Steam/steamapps/compatdata/244850 > find . -iname '*DiaSym*'                                    
./pfx/drive_c/windows/Microsoft.NET/Framework64/v4.0.30319/diasymreader.dll
./pfx/drive_c/windows/Microsoft.NET/Framework/v4.0.30319/diasymreader.dll

I am not sure what exactly adds it, I usually install dotnet through winetricks (specifically the dotnet48 package).

I cannot access the game myself at the moment either, but there is another easy way to repro without tinkering: Start a new game, select the "First Jump" scenario. The game should load but you will just be stuck in a box above the Moon. The reason is that scenario events are also implemented as C# scripts in the same manner as mods, so it will fail to compile and the scripted ingame cutscene won't load.

If you hit Alt+F11 you should get a debug menu and load errors in that in bottom right. Sadly, all exceptions are missing in there, as they are in the game log. The only useful info I managed to get is by running it with PROTON_LOG=1 %command% command line and examining the Proton logs.

P.S. - I did try compiling the dev branch from source and I still needed to patch the game. Might be something I did wrong, I'll try the CI build later.

[madewokherd]

But the log says Microsoft.DiaSymReader.Native.amd64.dll.

[madewokherd]

Seems like Microsoft.DiaSymReader.Native.amd64.dll may be the name of a newer version of the dll. If that's open-source, and a compatible version was in .NET Framework as diasymreader.dll, then we should be able to include it.

If not, we may have to reimplement it.

[madewokherd]

This suggests it's not open-source: https://github.com/dotnet/core/blob/566636c496b154010f06e51c36ef6f8f92683b00/license-information-windows.md

And I couldn't find any source code for it. It does appear to be part of the unmanaged .NET Framework API. Since it's an unmanaged component, it's unclear whether it should be part of winehq or wine-mono, but either way we'll need a replacement.

But we should first figure out what Space Engineers' usage is so we know what to implement.

[madewokherd]

My guess is that it should be part of Winehq, since it likely has a stable interface that wouldn't require any other part of wine-mono to function.

[Onyx47]

Thank you for all the research either way, I am not knowledgeable enough on .NET internals to have found this on my own.

I'll try and dive more into the game how exactly it reaches the codepath that calls diasym because I don't remember I saw it referenced directly. I am a bit at the edge of my understanding of dotnet here but if I come up with anything useful I'll post an update. If you think the issue is at a bit of a deadend and don't like to keep stuff like this opened close it and I can open a new one if I find any new info.

[madewokherd]

I don't think it's at a dead end, I'd like to investigate it directly when I get a chance.

[madewokherd]

"First Jump" isn't listed under Scenarios for me.

[Onyx47]

This is one of the default scenarios and should be present:

Alternatively, to try with a mod:

• Go to Options > Game > Tick Experimental Mode
• Click on New Game, click on Custom tab, you can select any of the worlds, "Empty world" might be the best option for testing purposes since it loads the fastest but otherwise it does not matter which you pick
• Click on Mods
• Click on Browse Workshop on the bottom
• Click on the little square in bottom right of the mod to subscribe to it. I'd recommend "BuildInfo" which should be on front page at the moment and is likely to stay there too (if it's not, search is available in top right). It's rather small and contains scripts instead of just model swaps etc.
• Close that window now. The mod should appear in the list on the left. If it doesn't you may need to click Refresh
• Double click the mod to activate it, should appear under Active mods now
• Click Ok
• Click Start
• The world should load up. If you chose a world type that asks where you want to spawn, just spawn where ever.
• Hitting Alt + F11 should show you a debug menu and you should have a mod error listed

For future starts you can just hit Continue from main menu, since mods are updated (if needed) and recompiled on every load.

[madewokherd]

Oh, that was actually the first thing I tried, but I didn't know there were supposed to be scripted events.

[Onyx47]

Yeah, if you run it using MS dotnet in Wine or on Windows it should play a cinematic and spawn you somewhere else after it's done playing.

Either way, I can also confirm it starts OK on my end with the latest CI build, I must have done something wrong on my end when compiling from source. The errors with mods are identical however.

Also, in case you want to check the game logs as well, they are stored as <PFX>/drive_c/users/steamuser/AppData/Roaming/SpaceEngineers/SpaceEngineers_YYYYMMDD_HHMMSS.log (Steam game ID is 244850 for Proton prefixes). Sadly, other than saying it failed compiling it's rather silent (search for "Mod Error").

[madewokherd]

I'm noticing a few other issues: Excessive load times, probably caused by those web timeout exceptions, and videos failing to play because CLSID_CWMVDecMediaObject isn't loading.

Here's the code that tries to load disaymreader: https://github.com/dotnet/symreader/blob/dd6b71744be8cdbfd69117e96f5b9e64a7b38b6e/src/Microsoft.DiaSymReader/SymUnmanagedFactory.cs#L155

It falls back on loading via CLSID 0AE2DEB0-F901-478b-BB9F-881EE8066788. Looks like it wants this interface: https://docs.microsoft.com/en-us/dotnet/framework/unmanaged-api/diagnostics/isymunmanagedwriter-interface

But, it uses Marshal.GetTypeFromCLSID instead of Type.GetTypeFromCLSID, and for some reason in Mono Marshal's version just throws PlatformNotSupportedException. That's easily fixed, and once it is we get a failure to load the CLSID.

As long as they don't try to read the debug info, we can probably get away with stubbing it.

[Onyx47]

Load times are just long in the game no matter what honestly, though I don't really have a side-by-side benchmark with Windows to know if it's much longer on Linux though. Some of the startup is also probably the intro movie failing to load, adding -skipintro as a parameter to the game helps with that at least.

As for debug info, I am really not 100% sure, it might be used in the modding build but that's more niche, and modding is still possible even without using that build.

I am sadly slightly lost here since all of this info is way out of my domain, but if I can help in any way do let me know and I'll follow whatever instructions to the best of my ability.

[madewokherd]

That's fine, I just like to work in the open when I can.

[madewokherd]

I think I have this working (even the cutscene plays), just need to clean it up and submit it. Thanks for reporting it.

[Onyx47]

Hello, just to check if I got this right so I can update the community and help with more testing if I can - this also requires an upstream wine patch, the newer releases are not enough? Is it just a missing a DLL override or is there more that needs to be done?

[madewokherd]

Yes, Wine patches are required, and I haven't sent the full series yet. I guess I can attach them here for testing.

[madewokherd]

Here's the full patch series for this. diasymreader-patches.zip

[madewokherd]

For testing purposes, it's important that the Wine patches are set up before the new Wine Mono package is installed, so that the Wine Mono package can install and register the new diasymreader.dll.

[ghost]

First attempt with wine git + the diasym patches, it crashed on an unrelated issue when loading a mission, e.g DX11 related. Wine-7.1-git + patchset - Learning to Survive

Second attempt with latest DXVK included, loads fine into a mission. Was able to play around for a few minutes with some hard stutters, then it crashed with a probably unrelated SQLite issue. Wine-7.1-git + patchset + DXVK - Learning to Survive

Third attempt, loaded into a different mission just to make sure the SQLite crash wasn't a onetime fluke. Played for a few minutes, lo and behold the same crash occurred. Wine-7.1-git + patchset + DXVK - The First Jump

If there's anything else I can do to assist, please let me know!

[Onyx47]

@fureloka out of interest, have you tried a custom game with a mod that requires scripting support as well, or just scenarios? I wasn't even aware SE uses SQLite for anything, and this might be a scenario-specific issue. Would be nice to know if so.

I'll try to compile Wine or even ProtonGE myself, just so we can see if it's maybe something that already got fixed in Proton but never made it upstream, didn't get around to it yet due to RL stuff.

As for stutters, it's probably garbage collector issues similar to what we had with MS .NET as well, I experienced some of that as well when testing but figured that's something that can be investigated once the game actually works fully.

[ghost]

Nope, only the first two scenarios, as I'm new to Space Engineers it was the most natural choice. Any specific mod you have in mind?

A note if you choose to compile Wine from git, you will have to fiddle a bit with the patches. Some of them have made it into the main repository, others haven't yet.

[Onyx47]

I just realized that I haven't read the log carefully enough: the crash seems to be caused by whatever analytics Keen is using. Apparently there is a plugin floating about that disables it, also not sure if removing GDPR consent would be enough, since I was running a game without mods just fine without the patches without doing anything special regarding analytics.

Maybe try going to Options > Game and unchecking GDPR consent if it's checked, see if it helps.

Either way, I don't think this has anything to do with the patches here and is a separate issue.

[madewokherd]

The diasymreader patches have been merged, and will be in Wine 7.2.

[madewokherd]

Unfortunately the logs don't really tell me anything except that it crashed calling sqlite3_step from sqlite3.dll. But, since the sqlite3 source code is available, (https://github.com/sqlite/sqlite/blob/63a47336afb8ae8f3e4a48b5f5a19201b1f04e13/src/vdbeapi.c#L761 for my own future reference), with some further logging I might be able to figure out what's wrong.

Can you get a log with WINE_MONO_TRACE=x WINEDEBUG=loaddll,seh?

As for stutters, it's probably garbage collector issues similar to what we had with MS .NET as well, I experienced some of that as well when testing but figured that's something that can be investigated once the game actually works fully.

Do we know for sure that this is garbage collector related at all in either case? That it would behave the same between .NET and Mono on Wine makes me suspect that it's some other issue.

You can try setting the GC_DONT_GC variable to check, but of course without garbage collection it will leak memory and probably eventually crash.

[ghost]

Not sure if the SQLite crash got fixed somehow, as I'm unable to reproduce it. One DLC (Warfare 2 Broadside) and two hotfixes has been released since, may explain why. Another plausibility, it could have been a bad library on my side that caused it...

Anyways, ran the game with Wine git, latest DXVK and a fresh prefix. The first time setup ran as usual, but SE wouldn't run. Looked at the logs, observed that MS .NET was being used. Uninstalled MS .NET and Wine-Mono, then reinstalled Wine-Mono, this restored the prefix back to a proper state.

Is MS .NET supposed to override Wine-Mono?

Log: Wine 7.2-git + DXVK - First time setup

With a proper prefix, SE was now able to start. Played through mission 1 of "The First Jump" scenario (No SQLite crash). Mission 2 on the other hand, crashed the AMDGPU driver. Repeated the scenario 3 times, it would always crash the driver right after loading into mission 2. Looking at the Mesa issue tracker, this seems to be known: Mesa - Issue #4165. I'll post to Mesa, when I have proper debug logs. Log: AMDGPU - Timeout

Decided to test Proton 7.2 GE-2. The MS .NET override did not happen, it did not stutter, but the AMDGPU driver crash did however happen at the same location.

As for the stutters, it wouldn't surprise me if it were caused by steamwebhelper.exe, it's crashing and restarting every two seconds.

[madewokherd]

You can't have .NET Framework and Wine Mono installed at the same time. They have different versions of the same files so only one can work.

[Onyx47]

Sadly I didn't manage to try the patches before, got sick and then work became crazy... but, I can confirm that everything seems to work here including plugins (which are not a thing many people use and are not even pushed by the devs) with Proton 7.2-GE-2 after I removed the protonfix for SE so it didn't force a dotnet install! I guess I should open an issue there to remove the dotnet install after some more testing because it does seem like we don't need it any more :)

@fureloka - was the crash on a planet and do you have trees and grass enabled? There is apparently a bug that only happens with AMDGPU where vegetation crashes the game. It is not related to .NET or Mono.