Epic: Investigate FOMOD support in the project

[halgari]

A lot of Bethesda games use FOMOD installers for their code. Vortex interfaces with this code via https://github.com/Nexus-Mods/fomod-installer but does so via IPC calls. The IPC interface is partly for security reasons (some FOMOD installers contain C# code), and partly because the installer is written in C# and Vortex is written in Typescript.

Using IPC and serialization seems like a lot of work for installers that contain only declarative XML installers. And it seems like there should be a better way to sandbox C# scripts but this has been extensively investigated in the past so we may already be using the best option.

Requirements

• [ ] Can execute fomod installers via invoking a IModInstaller
• [ ] If possible, record options selected for use in collections (store it in the mod metadata)
• [ ] Code for executing the installers should be in the main project, usable by Skyrim/Fallout/Oblivion support in the future.

[erri120]

Isn't FOMOD (C#) for Fallout 3 and NV only, similar to OMOD, which is only for Oblivion? The game specific mod managers Fallout Mod Manager (FMM) and Oblivion Mod Manager (OMM) are C# mod managers and both have a mod format that allows for embedded C# scripts (they also share creators).

FOMOD (XML), which is commonly used for Skyrim and Fallout 4, is also a hot mess because there is no real standard and older mods would still have to be supported.

[halgari]

All true, so that's another question: can we split out the logic for the older installers that require sandboxing, and keep that as IPC, while bringing the newer XML installers into the application code?

[erri120]

Another issue both FOMOD (C#) and OMOD (C#) scripts have, is the fact they can build UIs using Windows Forms. Both FMM and OMM were originally .NET Framework 2.x projects. OMM got abandoned and FMM continued development on GitHub: https://github.com/foesmm/fomm however, it's still stuck with .NET Framework 4.x.

[TanninOne]

Bit of an info dump that will mostly just repeat what you already said:

OMOD is a custom archive format containing the mod + a script file written in a DSL (ModScript) and afaik was used only with Oblivion. FOMOD is a regular archive containing the mod + either an xml, c# script or ModScript script and was used in FO3, FO:NV, Skyrim and other games. Apart from a few game-specific functions in the api for the script languages it's reasonably universal.

ModScript is probably secure enough simply by merit of being restrictive enough as a language and since the DSL exposes its own interface functions and scripts don't call windows.forms directly, porting it should be possible. I'm not aware of any mod actually using ModScript with fomod so this this purely hypothetical.

The c# scripts are distributed in source code form and are compiled on the users system. They import their own libraries (including windows.forms). With Vortex we've recently updated from .NET framework to .NET 6 and so far there were no complaints about compatibility issues so at least moving to .NET core is possible without breaking those installers.

The sandboxing was originally done with sandboxed AppDomains running in the same process as NMM, the scripts would directly drive file installation and such. With Vortex we initially kept that sandboxing mechanism when moving to Vortex but the installer is in a separate process and delegates any actual filesystem opertaions to the main process using either a pipe or tcp. The appdomain isolation is not supported in .NET non-framework and Microsoft has been advocating against using the functionality, they didn't seem to trust it themselves so when moving to .NET 6 we had to find a different solution.

So with the move to .NET 6 we instead run the entire installer process in an "App Container" (CreateAppContainerProfile api). It's the mechanism UWP is being run. I think it should be more robust than the previous library solution and better maintained since MS themselves make heavy use of it. Originally, in NMM, the sandboxed appdomain allowed filesystem access and such to directories that the installer might reasonably need access to. With Vortex, the sandboxed process has almost no permissions because it has to go through the main process anyway.

The current solution does suffer the issue that there are many ways this could be broken on a user system. E.g. it requires certain (default) permissions on c:\program files, otherwise the sandboxed process can't read the .net installation, also certain services (like "RPC Endpoint Mapper") have to be running (again: default) It seems to be <1% of Vortex users affected but at our scale that's still plenty. That's why Vortex also has the option to disable the sandbox, the assumption being that if anyone did actually release a malicious installer, the 99% users with working sandbox would make us aware of it quick enough for it to not be a feasible attack vector.

It might be an option to just be more radical and drop the feature entirely. Inform the community that no new c# scripted installers will be supported, identify all existing installers, create a dictionary of the hashes of the script.cs files. Then either review and precompile them all into a single assembly of supported installers and then when someone tries to install one of the mods with a recognized script.cs file, run the corresponding code from the previously verified assembly - without a sandbox because that code is trusted. Or: write xml based installers to replace the scripted ones and run those instead. More work (but may not be that much because I don't think there actually are a lot of scripted installers) but now NMA can integrate them better (both in terms of UI and other features, e.g. now you can track with installer options the user picked). AND if we go the latter route, these replacement installers would also be available to mod managers that don't run c# and don't want a c# component.

Actually, I've always thought it could be a really neat feature to have the option to host installers (xml fomod or whatever) separately from the mods. Identify the installer by a checksum or whatever, if the installer repo has something for it, use that. Could help mod managers that don't support certain installers or with weirdly packaged mods, ...

[erri120]

For the OMODFramework I created a while back, I implemented the OBMM scripting language and just inlined the C# scripts. I manually went through the most downloaded Oblivion mods and only found 3 mods that had C# scripts. The library was integrated into MO2 almost a year ago, and no one has complained about broken C# scripts yet, although it's totally possible that nobody is even using this. The only real issue that remains when inlining scripts, is the WinForms import which forces netX.0-windows.

Converting the C# scripts to another format might be viable. At least for the 3 C# scripts I found, DarNifiedUI and DarkUIdDarN mostly just use C# to create a “sexy” installer UI with WinForms, while HorseArmorRevamped uses more advanced features and does binary editing.

It might be worthwhile doing a survey of at least the top 100 or 1000 mods of each game to figure out which format is in use. OBMM also supported Iron Python and VB.NET scripts, but I haven't found a single mod that uses it, so I didn't implement it.

[Sewer56]

I don't have much to say at the very current moment; regarding the situation as I'm from a non-Bethesda background and thus I would need to get my hands dirty before I can think of any sound design decisions. e.g. I don't know the extent of functionality supported by the XML format.

That said; a few things come to mind:

• We will probably need to split this up into an Epic.

  • We added the appropriate templates/epics into the NMA repo today.
  • There's multiple formats/installers involved for the mods; each requiring specific support.

• There are some serious concerns with cross-platform support for the C# variant.

  • We cannot use WinForms/WPF for obvious reasons.

  • Supporting UIs would require us to use Roslyn to rewrite any WinForms related code, e.g.

    • Produce equivalent AvaloniaUI controls for every WinForms control [which would be then mapped onto a Canvas control].
    • Ensure there are no remaining references to WinForms.
    • This conversion would be lossy; e.g. any super custom logic tied to a control would be lost.

  • Controls will probably vary in size between our theme and the WinForms originals, potentially leading to awkward situations where items might look misaligned; etc.

  • Any mods that tried to extensively theme the original Windows Forms might end up looking alien as it's not feasible to support all properties of WinForms.

  • Sandboxing in general is difficult.

  • As noted, even Microsoft gave up on AppDomains; instead spinning its most useful feature of DLL/plugin supporting isolation to AssemblyLoadContext.

  • While it is possible to restrict API access by either:

    • Running a sub-process in some virtualised container.
    • Playing constant whack-a-mole with Harmony to temporarily restrict/disable certain APIs.

    It is probably not worth the investment effort, as we now would have to support multiple platforms.

    On Windows you can try CreateAppContainerProfile API etc. but then...

    Linux and (probably) OSX this would get pretty complicated real fast: You'll suddenly need to pull a dependency on some package in the native package manager [whose name will vary by distro]; otherwise if we use FlatPak/Snap/AppImage etc. we'll have to pray that whatever component we need can be easily pulled in via an existing runtime/dependency; otherwise we're going to have issues.

  • There's that <1% of users with issues, even on Windows. Ideally they shouldn't have to run into any issues in the first place.

Misc Note: Technically speaking @TanninOne is probably the most qualified for F(OMOD) related stuff as they have already worked with it in the context of Vortex.

[Sewer56]

On that note, since sandboxing was brought up. It's something that's probably worth discussing as a general topic on its own at some point.

Although it may potentially be a bit of an unpopular opinion, I generally find the idea of implementing sandboxing as a security feature in the context of *game modding* to be pretty moot. I think 'security' in the context of mods as more of a moderation issue; much like of how you can think McDonald's to be a Real Estate business rather than a Restaurant.

---

Usually this varies by developer/game studio but most games I've historically dealt require code injection to get any decent modding support at all [i.e. anything more than an asset swap].

There are some exceptions of course, e.g. RPGs often use custom scripting languages and you can get plenty done with that; but most games require hooking and direct code edits to get anything significant done [e.g. Multiplayer]; and almost all old games require specialised patches for e.g. Widescreen Support, XInput Support etc.

Outside of the realm of native games; all mod loaders for games .NET (Mono, Unity IL2Cpp, CoreCLR) that is BepinEx, MelonLoader, MonoMod & Co. are based on arbitrary code execution and would live outside of our launcher when the end user opens the game.

Even if we were to theoretically 100% bulletproof secure the launcher; there's generally nothing stopping authors from putting potentially malicious logic in their Native and/or Managed .NET DLLs that would be deployed by our launcher. Hence I believe that to be more of a 'moderation issue' as realistically speaking we cannot feasibly control every single piece of executable code that makes it to nexusmods.com.

We can only moderate it; e.g. request source code is publicly available for individual mods.

---

Truth be told; I'm not sure how to best express the message I want to deliver [it's a bit difficult].

But in a way it goes back to the age old security vs convenience debate. Advanced game modding is ultimately based on 3rd party arbitrary code execution and there will always be potential security risks associated with that. You'll always be playing whack-a-mole and can't possibly fully control every single .dll, .asi, .so etc. file ever uploaded to the site; hence to me it's more of a moderation issue. If you don't get pwned in-launcher during the deployment process you'll get pwned at runtime.

As such, at least in my opinion: It's better in general to take the performance/convenience tradeoff *in the context of game modding*. I'd much rather focus on the UX and get the deployment process done in <200ms than having the end user to wait 3-5 seconds to launch a whole separate process; read DLLs from disk again, re-JIT all the code and validate every single read/write via RPC.

If it came down to it and we had to do something [that's more than nothing]; I would much rather warn the user of 'arbitrary code execution' ahead of launching game/script. Though realistically, most end users just tend to ignore these dialogs.

As for Sandboxing and/or FileSystem Virtualisation itself, I personally see its use more as a potential compatibility feature (rather than a security one) should it become potentially necessary or generally useful; e.g.:

• You want to be able to avoid potential garbage files [logs etc.] made by a script.
• When legacy scripts expect certain conditions/constraints e.g. needs read access to certain data; expecting to be used within a specific environment.

[halgari]

I think we have several paths forward on the topic of sandboxing:

• We can not support mods with .cs files and demand that users update them
• We can support them in a sandboxed mode of some sort
• We can add them to a curated "allow list" that runs unsandboxed. Essentially we'll ever only run files of a specific type
• We can hand-code alternative installers, perhaps initially shipped with the app. This ends up being an "allow list" of sorts

What's interesting is that the first and last approach result in not needing sandboxing, an external .NET instance or any sort of SDK (csharp compiler) installed on the user's box.

As far as the security risk goes, I think it depends a bit on where the code is being executed. People understand that installing a mod and then running the mod could break their system, but I think they're a bit unprepared for the idea that opening a mod in an app could brick their system. So while you're right that the end risk is the same, I think this has a higher chance to catch users off-guard.

[Sewer56]

Ideally down the road; I think using a combination of the options would probably be for the best. Here's one possible approach from many:

• Support a subset of FOMOD functionality.
  • If it doesn't turn out too hard, auto convert very basic forms to Avalonia.
  • I don't think any of us is likely to have experience with Roslyn [the C# compiler]; so not 100% sure of how difficult it'd get.
  • Anything that requires advanced UI features (e.g. theming) and/or may break; convert by hand.

As for the security side; perhaps we could start with hashing the original .cs installer file and...

• If it's on a curated whitelist of hashes, install without user intervention.
• If it's not on a whitelist, display an 'Arbitrary Code Execution' warning that the user can't immediately dismiss frame 1. [Ok button with 5s cooldown before dismissable ahead of running all scripts?]

Do note; having some numbers here could potentially be very useful.

Writing code that decently auto-converts WinForms layouts to controls in Avalonia canvas might take considerable time.
For instance, if there are less than 40-ish mods with .cs files that use WinForms; we could just consider hand converting all of them ourselves; as that could still take less time than getting down and dirty with Roslyn.

[halgari]

I think that last sentence is the method we want to go down. I'll convert this into an epic and split out the different parts. I think we can safely punt on all but the XML support for now. XML FOMODs give us support for Skyrim, SkyrimSE and Fallout 4, which are by far the most popular Bethesda games. We can get FOMODs working for them, get the games supported and working well, then loop around later to figure out how to solve the OMOD and csharp installers.

I think it's highly likely we may be able to say "if you have a installer that has a .cs script in it, update your mod, install the files by hand or use a different mod". We have to have some standards after all :D

[TanninOne]

Writing code that decently auto-converts WinForms layouts to controls in Avalonia canvas might take considerable time. For instance, if there are less than 40-ish mods with .cs files that use WinForms; we could just consider hand converting all of them ourselves; as that could still take less time than getting down and dirty with Roslyn.

Exactly this. I don't have any real numbers but I suspect the number of .cs script installers is probably in the dozens not the hundreds. I think of these two variants (proxying the ui code vs. writing replacement installers), the replacement installers are the much more practical solution because it will be less work (more repetitive but less challenging), be more robust and we might have further use for it down the line (e.g. as a way to "fix" other broken mods/installers when the original author isn't available any more or doesn't want to re-upload a 2GB mod to fix a typo in the installer script). I think generally, having the ability to have the installer separate from the mod data could be a really cool feature in general.

We can not support mods with .cs files and demand that users update them

There is a good chance that for many of these mods the author isn't even active any more, you'll never get them all to rewrite.

[halgari]

Investigation is done, further work will be done in other tickets