search

HaloSPV3 / Gemini

Configure SPV3 with this flavorful GUI frontend for HXE.
zlib License
6 stars 2 forks source link

Linux compatibility #54

Open BinToss opened 2 years ago

BinToss commented 2 years ago

NOTES

DEPENDENCIES, RELATED ISSUES

RELATED LINKS

KEY ISSUES

DEPENDENCY: MahApps.Metro

This GUI framework aids in the creation of Metro-styled GUIs. It is dependent on Window Presentation Framework. This will be especially difficult to resolve.

Solution

Migrate from WPF to Avalonia

old proposals A. Try mingling [UNO](https://platform.uno/) with our dependencies, though that probably won't resolve MahApps.Metro's WPF dependency nor WPF's dependency on Windows runtimes. B. Completely migrate away from MahApps.Metro and WPF to [Avalonia](https://avaloniaui.net/), [GTK#](https://github.com/GtkSharp/GtkSharp), [QtSharp](https://gitlab.com/ddobrev/QtSharp), [Qml.NET](https://github.com/qmlnet/qmlnet), or [UNO](https://platform.uno/).

Windows Registry

See HaloSPV3/HXE#217

Steam

Pseudo-DRM was dropped. No more Steam issues...for now. ## Path inference of MCC based on location of Steam.exe Unix OSes don't use the Win32PE EXE format. As such, there won't be a Steam.exe to locate. ### Possible Solutions A. If `System.Environment.OSVersion.Platform` returns `PlatformID.Unix`, prompt the user to locate a 'Steam' assembly without requiring a file extension. B. Allow the user to input the path to 'halo1.dll'. This would require the least amount of development time, but it forces more work upon the user if they choose this method to validate ownership of "Halo CE". As usual, the frontend would be implemented in `SPV3` using the [_pre-existing_ backend](https://github.com/HaloSPV3/HXE/blob/main/src/MCC/Halo1.cs) in `HXE`.

ROAD PLAN

There are two routes we can take to resolve this issue.

STOPGAP MEASURES

Platform-dependent code paths will be determined at runtime. This implies the following:

MS Docs Example: ```cs // This example demonstrates the PlatformID enumeration. using System; class Sample { public static void Main() { string msg1 = "This is a Windows operating system."; string msg2 = "This is a Unix operating system."; string msg3 = "ERROR: This platform identifier is invalid."; // Assume this example is run on a Windows operating system. OperatingSystem os = Environment.OSVersion; PlatformID pid = os.Platform; switch (pid) { case PlatformID.Win32NT: case PlatformID.Win32S: case PlatformID.Win32Windows: case PlatformID.WinCE: Console.WriteLine(msg1); break; case PlatformID.Unix: Console.WriteLine(msg2); break; default: Console.WriteLine(msg3); break; } } } /* This example produces the following results: This is a Windows operating system. */ ```

BUILD FOR EACH PLATFORM

Multi-Platform Alternatives to WPF/WinForms

Formerly tracked in HaloSPV3/HCE#249.

ThisNekoGuy commented 2 years ago

Linux Note:

Every game installed through Steam (and by extension, Proton) installs in separate Wine bottle / prefix, so searching the registry via Proton will produce no results as every instance is isolated.

For Proton, either there will need to be a solution for detecting the MCC bottle/prefix some other way or the user may have to target it themselves until such a solution is found.

For plain Wine, this shouldn't be an issue (don't quote me on that) because the user would already know to run the installer through the associated Wine bottle/prefix themselves in order to detect the installation of either Windows Steam or an original CE install.

BinToss commented 2 years ago

Every game installed through Steam (and by extension, Proton) installs in separate Wine bottle / prefix, so searching the registry via Proton will produce no results as every instance is isolated.

That's...stupid. That breaks apps that depend on other apps' Registry entries.

btw, I have to confess that Linux compatibility is and always has been a low priority because it requires a huge rework. Bug fixes, UX, and new features for currently supported platforms are typically normal-priority and high-priority issues.

However, even those are a low priority after having migrating to .NET 5 a few months ago. The decision to ditch .NET 4 because its SDK irritated me has come back to haunt me.

Any new decisions involving the installation procedure will not make it in time for SPV3.3.1 which has been delayed far too long.

ThisNekoGuy commented 2 years ago

That's...stupid. That breaks apps that depend on other apps' Registry entries.

That said, the main reason Valve has it set up this way is because not every game has the same dependencies and, if something goes wrong with a prefix, they have them separated to minimize the potential damage it might have on games that would theoretically share that prefix (like using different Proton versions for specific games because that "updates" the prefix)

btw, I have to confess that Linux compatibility is and always has been a low priority because it requires a huge rework.

That's understandable, it's the case for many projects; usually though, that's just incentive to think about compatibility from the outset rather than an after-thought for future projects because it avoids anyone having to do extra work later

On a related note, MAUI seems to not support Linux at the moment - it brought MacOS support first:

EDIT: GTK might be an option, because it's a known working toolkit that's cross-platform, but I'm hesitant to consider it given that, on Linux, it's appearance/presentation is very biased towards Gnome features, including app title bars and file manager windows - disrespecting those provided by the system/user defined desktop environment settings (such as: KDE Plasma, LXQt, etc).

That aside, GTK# has another potential problem, functionally: it might not be compatible with .NET 5

As for packaging a theoretical native installer for distribution, this would be best done with an AppImage; they have self-updating features, don't have permission issues associated with other distro-agnostic packaging formats, and can be portably used on any drive - mitigating risks, also presented with other packaging formats, that could potentially eat an OS's drive space

SubhadeepJasu commented 2 years ago

The gtk appearance is a non issue. I can make it look like anything and force it to use that look cross platform. I'll say let's start with it without MCC validation and once we at least got the game working and installing then we can look into it.

ThisNekoGuy commented 2 years ago

That's good, then

I'd be down for GTK but the main issue here is that it would mean switching to a toolkit @BinToss would be unfamiliar with at the moment Though, at the same, it's better for development compatibility because XAML projects are basically locked to Visual Studio and GTK# isn't (which is easier on me as a Linux developer and mostly depends on .NET Standard 2.0 for consistent cross-platform compatibility)

That said, going backwards after just moving to .NET 5 might sound frustrating, especially when I know Bin was interested in moving to .NET 6 for MAUI to find out that it isn't cross-platform with Linux yet :/

If GTK is decided, I'll look into checking out the toolkit

BinToss commented 2 years ago

MAUI seems to not support Linux at the moment

So our current options for a multiplatform GUI is limited to GTK or UNO. Miris and I have been discussing the plausibility of providing CLI apps in addition to the GUI apps. However, the potential for feature disparity–one interface's features becoming more rich than the other's–is discouraging.

That said, going backwards after just moving to .NET 5 might sound frustrating, especially when I know Bin was interested in moving to .NET 6 for MAUI to find out that it isn't cross-platform with Linux yet :/

Luckily, netstandard2.0 and netcore3.0 are both forward-compatible with net5.0 and net6.0. Probably net7.0, too. Yes, you read that right: net6.0 isn't stable, yet net7.0 already has preview releases. I considered back-porting to netstandard2.0, but...

  1. The use of C# 9 language features would need to be reverted.
  2. We would be unable to use net5.0+ compatibility packages that allow .NET Framework dependencies to be referenced in non-Framework projects.

net6.0 will have fewer benefits for our projects due to delays in development of WPF's compatibility with IL Trimming and MAUI's lack of support for Linux. The former has been delayed to net7.0.

BinToss commented 2 years ago

I've added a new header to the OP titled "Multi-Platform Alternatives to WPF/WinForms" with our current suggestions and some new ones.

Multi-Platform Alternatives to WPF/WinForms

ThisNekoGuy commented 2 years ago
  • Qt (QtSharp or Qml.NET)

I didn't even know Qt had .NET bindings :open_mouth: And since GTK# and Qt themselves don't depend on XAML, it would honestly make working on the project easier on non-Windows systems... (e.g. such as my PC; since XAML is Visual Studio exclusive :upside_down_face: - Thanks, Microsoft.)

BinToss commented 2 years ago

image MAUI won't support Win7, so I'll be looking into UNO next. They're similar in the sense that they're multi-platform wrappers for native UI libraries, except UNO supports Windows 7. Also, UNO allows XAML design which will (hopefully) make porting the GUIs a bit easier. Uno also provides an XAML extension for VSCode that introduces the features you'd expect in Visual Studio IDE.

On Linux OSes, developing with UNO is rather limited in regards to target platforms when compared to macOS and Windows. See Target platform coverage by IDE on Linux. It's possible to target Win7+, but debugging and hot reloading is restricted. Probably because it requires Windows OS runtimes.

BinToss commented 2 years ago

Does Steam's executable on Linux use the ELF file extension or no extension? Scratch that. Better question: how can we infer MCC's path via Steam on Linux distros? The current method... ...locates Steam's root directory via Steam.exe or a .LNK shortcut to it ...searches Steam's directory for LibraryFolders.vdf, the file that lists all of Steam's local library paths ...searches all of the discovered libraries for "./steamapps/common/Halo The Master Chief Collection/halo1/halo1.dll" (maybe we should be looking for ".../halo1/maps/a10.map" to ensure Campaign ownership)

BinToss commented 2 years ago

Everything related to DRM (including MCC CEA detection) will be removed from or skipped in SPV3.3.1. SPV3 is going completely free.

This means...

ThisNekoGuy commented 2 years ago

~Does Steam's executable on Linux use the ELF file extension or no extension?~ Scratch that. Better question: how can we infer MCC's path via Steam on Linux distros? The current method... ...locates Steam's root directory via Steam.exe or a .LNK shortcut to it ...searches Steam's directory for LibraryFolders.vdf, the file that lists all of Steam's local library paths ...searches all of the discovered libraries for "./steamapps/common/Halo The Master Chief Collection/halo1/halo1.dll" (maybe we should be looking for ".../halo1/maps/a10.map" to ensure Campaign ownership)

The game's location (via SteamID) can be found in: ~/.steam/steam/config/libraryfolders.vdf So, pretty much the same method should work (though note: case sensitivity matters)

As for the DRM being dropped, I'm a bit surprised; I hadn't expected that - why drop MCC's detection? And how does dropping the check for ownership protect SPV3?

BinToss commented 2 years ago

Regarding GUI, Avalonia is the most appealing atm. the https://github.com/AvaloniaUI/Live.Avalonia NuGet package provides Live Reloading XAML on Linux distros, but it does not provide in-IDE XAML design.

For the past month or so, I've been rewriting the "Compress Install folder on NTFS" feature to use Kernel32 P/Invoke instead of WMI. Neither of these will work on WINE or Proton, so I may just disable P/Invoke calls to the Kernel32 and AdvApi during run-time. We also make some P/Invoke calls for the borderless hack, so that will need to be disabled, too.

ThisNekoGuy commented 2 years ago

For the past month or so, https://github.com/HaloSPV3/HXE/pull/292. Neither of these will work on WINE or Proton, so I may just disable P/Invoke calls to the Kernel32 and AdvApi during run-time.

Wine can be detected at runtime, for the Win installer, so you can just detect and disable the NTFS code; I actually know a project that did this recently, though not due to NTFS specifically: https://github.com/neatodev/BmLauncher/blob/master/data/WineChecker.cs

Though, for a native installer, that code would have to be avoided, yeah :p

ThisNekoGuy commented 2 years ago

Regarding GUI, Avalonia is the most appealing atm. the https://github.com/AvaloniaUI/Live.Avalonia NuGet package provides Live Reloading XAML on Linux distros, but it does not provide in-IDE XAML design.

As far as I'm aware though, there aren't any XAML design options for IDEs outside of Visual Studio (non-Windows exclusive IDEs); it's still nice it has live-reloading but we're not going to find a Linux XAML designer anywhere until some 3rd party is actually bold enough to make one, unfortunately :/

BinToss commented 2 years ago

On top of .NET Framework 4.8 Runtime's poor quality on WINE, it doesn't support the C# 9 extensions present in some CsWin32 generated code. Only .NET 5 and up are guaranteed to support these extensions. For testing purposes, I'm targeting net462 while overriding its C# 7.3 limits with C# 9 targets, but anything can break unexpectedly this way because of the net4x Runtime's limits.

.NET SDK 6.0.300 can compile SingleFile (all-in-one) executables for Windows 7 SP1 which solves the loose, unavailable, or wrong-version dependencies problems, but I'll need to get rid of WPF and WinForms because they are trim-incompatible and add at least 50MiB to a Self Contained deployment. This is the reason why I'm looking into alternative GUI platforms.

However, .NET 5+ Windows executables don't run on WINE. I'll have to build and release separate executables for Linux and MacOS. This will require cross-platform GUIs, system-agnostic paths, WINE/Proton automation or user instructions, and...what else? Probably DLL-mod management.

ThisNekoGuy commented 2 years ago

Honestly, I wouldn't be too hard-pressed about MacOS support considering supporting OpenGL and Vulkan via anything (including Wine) is a nightmare and a half because of Apple; not going to talk you out of that, of course, just a heads up reminder I'm sure you might have heard of.

That aside, yeah, .NET5+ has had problems for a while with Wine; although, I know for sure native SingleFile executables are also do-able on Linux via .NET6, so that's cool. Wine instructions should be easy, until automation is implemented; system agnostic paths really just boils down to some if-checks and avoiding backslashes :p

The meat of this will mainly be the cross-platform GUI; the DLL-mod management will just be moving stuff in and out of the game directory, I take it, right?

BinToss commented 2 years ago

supporting OpenGL and Vulkan via anything (including Wine) is a nightmare and a half because of Apple;

I had completely forgotten about that. The only Mac user I know of is Masterz, the project lead SPV3 and Legacies. IIRC, he uses a somewhat older MacBook c. 2015. I don't know of any other Mac users.

Wine instructions should be easy, until automation is implemented; system agnostic paths really just boils down to some if-checks and avoiding backslashes :p

Yup. The hard part will be Wine/Proton management.

the DLL-mod management will just be moving stuff in and out of the game directory, I take it, right?

Indeed. The only data needed from a DLL is the version info and other assembly info, filename, extension, dates, and hashes for verification and file integrity. The loading of DLL mods will be handled by a mod loader DLL in the game directory (loaded by the game) and the rest in "./mods". Until Chimera 1.0 and OpenSauce incompatibility is resolved, we'll be Monolith mod loader 0.3.0 which replaces Halo's keystone.dll. Chimera 1.0 has newer versions of Monolith built-in and replaces Halo's strings.dll (normally contains exception strings and more).

Regarding the GUI framework, UNO has a cross-platform, out-of-editor visual designer provided by Figma. Meanwhile, the only cross-platform visual design available for Avalonia is via JetBrains Rider IDE. A Visual Studio Code extension is in the works, but is only available as a nuget package via nuget.avaloniaui.net. It does not yet have live preview or visual design.

ThisNekoGuy commented 2 years ago

Meanwhile, the only cross-platform visual design available for Avalonia is via JetBrains Rider IDE.

Unfortunately, I can't afford a JetBrains Rider subscription :skull_and_crossbones: Interesting though, that UNO has anything for that at all :thinking: I heard they were working on one but had been told previously it was for VS Code. As for the VS Code Avalonia extension... That would be great if it weren't incomplete Lol; I can't imagine it would be done anytime soon though :/

BinToss commented 2 years ago

https://stackoverflow.com/a/970134/14894786 Found a solution for DLLImport-related exceptions.

BinToss commented 1 year ago

@JediMasterChief notified me in Discord that they'll be documenting their steps for Lutris/Proton compatibility this weekend. Their method uses a fork of Wine specific to either platform: GloriousEggroll/wine-ge-custom, GloriousEggroll/proton-ge-custom

BinToss commented 1 year ago

Here are the steps I took to get SPV3 installed and running on Wine;

Use Winetricks, wine-ge-custom and a 64-bit prefix

  1. Install mfc42 through winetricks
  2. Install Halo CE and 1.10 Patch. You can simply double click if using default prefix, use the terminal to specify wine prefix to use before running the files, or run wine explorer via winetricks in the prefix of choice
  3. Through Winetricks, install .Net 4.6.1 or later and the arial font
  4. Create the directory you want to install SPV3 into via your Linux file manager, assuming you want any other location than default
  5. Run SPV3 installer using the same methods as step 2.
  6. Navigate to the location you want to install to via browse. Avoid clikcing on the text box. Give the installer a couple of moments to settle on that location before clicking install.
  7. (potetially optional) Once the Open Sauce step is reached, skip it and instead use the latest amaisosu from github. Install into the normal Halo CE folder or a copy of it to avoid overwriting SPV3's version of Open Sauce.
  8. Open winecfg, add SPV3's haloce.exe to the applications menu, then with that exe selected in that menu, navigate to libraries and add an native, builtin dinput8.dll override, and configure other options as needed.
  9. Follow the normal SPV3 setup from there.

Note: the instructions will not be valid for SPV3.3.1. Linux support will be in the form of native Linux executables where possible and a Mod Loader in the form of

BinToss commented 1 year ago

How do we ensure haloce.exe (or any app for that matter) runs on the highest-performance GPU on any given Linux distro? Would a specific app or library be needed to interact with drivers or does it have a vendor-agnostic settings panel similar to Windows 10 and 11?

P.S. load-balancing across all discovered dGPUs and iGPUs would be best, but is outside the scope of this project. The main goal for this specific query is to prevent the game from running exclusively on the power-saving iGPU without renaming the EXE to "halo.exe". Most drivers have a GPU-accelerated profile for "halo.exe", but not "haloce.exe". Changing the filename may have unexpected side effects.

ThisNekoGuy commented 1 year ago

@BinToss This is typically handled by the user; in hybrid-GPU scenarios, Optimus (Intel+Nvidia) would be invoked by the user either through Steam client launch options or with Lutris (non-Steam game launcher) https://wiki.archlinux.org/title/NVIDIA_Optimus

Typically, doing this through steam directly is more difficult when using Proton, specifically, because Steam's UI for invoking environment variables and telling Proton what to do is a bit of a mess; I'd advise launching the executable by using "PRIME render offload" somehow, either by the to-be native launcher doing this itself or with a bash script: https://wiki.archlinux.org/title/NVIDIA_Optimus#Using_PRIME_render_offload https://wiki.archlinux.org/title/PRIME#PRIME_render_offload

As for Intel CPU + AMD GPU / AMD APU + AMD GPU combo specifically, I can't say because I'm not personally familiar with it

I think the Linux GPU drivers should be acceleratated in discrete-only systems, otherwise, for the game (Though, do note that using the G-buffer setting will cause the game to suffer in performance on Linux)

ThisNekoGuy commented 1 year ago

Which reminds me, now that you ask, users should be encouraged to have gamemode installed, for better performance; we should also detect if this is installed (or include it ourselves) and load this library to keep performance up

BinToss commented 1 year ago

Related news: