Open YoshiRulz opened 4 years ago
Could we leave the net standard libs out of this? They're already net core compatible
I don't see us changing UI framework before .NET 5, so Windows and Linux will need separate "binaries".
We're already in the situation where the windows build and the linux build use completely different copies of system.windows.forms, yet things work with us only building binaries once. Why will that change?
Could we leave the net standard libs out of this?
Moved out of table.
Why will [build once, run anywhere] change?
Currently, the same binary can be used on Mono and Framework on Windows because it's the same TFM (net48
). Using net5;net48
will allow a binary for Windows, and a binary for Mono, both with WinForms support. net5
cannot be loaded by Mono, and Mono will never get Core 3 or .NET 5 support because the planned .NET 6 will unify the BCL implementations.
because the planned .NET 6 will unify the BCL implementations
Do you have a source for this? I've been looking and it's hard to find good information
[<praise of .NET Core>] .NET 5 builds on this work, taking .NET Core and the best of Mono to create a single platform that you can use for all your modern .NET code. [... .NET 5 will] expand the capabilities of .NET by taking the best of .NET Core, .NET Framework, Xamarin and Mono.
—Introducing .NET 5, 2019-05-06
Last year, we laid out our vision for one .NET and .NET 5. we said we would take .NET Core and Mono/Xamarin implementations and unify them into one base class library (BCL) and toolchain (SDK). [... Because of the pandemic,] unification will be truly completed with .NET 6, our Long-Term Support (LTS) release. Our vision hasn’t changed, but our timeline has.
—Announcing .NET 5 Preview 4 and our journey to one .NET, 2020-05-19
So I had a play with this thing (the Windows version works in Mono, who would have guessed) and the situation doesn't look too bad. GongShell, OpenTK 3, and SlimDX all reference System.Windows.Forms
, but should work in .NET 6 on Windows. IIRC natt saw that in practice when testing the and don't seem to work in .NET 6 on Windows.hammer-and-duct-tape
branch.
In other news, .NET 6 doesn't include WinForms for Linux, even though Mono currently supports it, and they're instead pushing ahead with MAUI (a.k.a. Xamarin.Forms). Given that, my plans for the Linux port are as follows:
hammer-and-duct-tape
branch. In effect, the current EmuHawk project becomes EmuHawkMono and a new EmuHawk which targets .NET 6 is created. In practice, the *Mono project files are new, and minimal changes are made to the existing project files.edit 2022-08-12: I've just been made aware of the ActiveQt library/framework which appears to require choice of Managed C++ or COM? Not sure if Mono supports either. edit 2024-03-21: Found this thing on HN which I think is ImGui but with vectors? https://github.com/rive-app/rive-sharp
Opentk 4.4.0 came out like 5 days ago and mentions joystick fixes. We can't update that on Linux from my understanding of this?
No and that's off-topic.
- Get tired of doing nothing. WinForms might not be getting removed from .NET, but it's also not getting any better. At the same time, all our library projects are stuck on Standard 2.0, unable to use some of the C# 8.0 and 9.0 language features. To reunify the binaries, we'll first need to decide if we want to switch to MAUI or to a third-party option (I'm only aware of Dear ImGui), and then focus on the one framework instead of dividing our efforts.
The other third-party option that was recommended to me several years ago was Eto, which translates to native options on their respective operating systems (WinForms, WPF, GTK, Cocoa): https://github.com/picoe/Eto The API very much resembles that of WinForms, which made it an easy choice back then. Although it seems Eto is still getting active commits, I'm not necessarily recommending it but rather just issuing a reminder that it exists.
I had started a crappy recreation of the WinForms UI in it several years ago, only to abandon it because I didn't want to spend the time necessary to recreate everything and moved back to WinForms before mostly suspending the macOS port entirely. https://github.com/Sappharad/BizHawk/tree/EtoHawk/BizHawk.Client.EtoHawk (That last commit from 2018 is actually a problem I ran into later with WinForms and ended up solving, but I don't think it affected Windows.)
SharpCompress cannot be upgraded past fixed in ae9135807, I was overcomplicating it0.26.0
—currently tracking as adamhathcock/sharpcompress#604.
tracking opentk/opentk#937 re: OTK3 keyboard init heisenbug; my OpenTK watchlist
FlatBuffers.Core is netstandard2.0-only, from what I can tell.
https://github.com/emclient/mac-playground has a port of Mono's WinForms to netstandard2.1. I'm not clear on whether it's compatible with Linux or just macOS, though.
That is just for macOS. It’s what I was using for the Mac port after Apple dropped 32-bit support. It won’t help you on Linux
OpenTK.GLControl was renamed to OpenTK.WinForms, which is in beta and supports netcoreapp3.1: https://www.nuget.org/packages/OpenTK.WinForms
Potential SlimDX replacement, targets .NET Standard 2.0: https://github.com/amerkoleci/Vortice.Windows
Supports Direct3D9, DirectInput, XInput, and XAudio2 among others, but not DirectSound. A few years ago I tried out the XAudio2 part and got it working in my forked repo: https://github.com/jdpurcell/BizHawk/blob/35ef85936666dbfdc1d6ad6bf87251f95cb51dbd/BizHawk.Client.EmuHawk/Sound/Output/XAudio2SoundOutput.cs
Vortice.XAudio2
doesn't seem to have a method for enumerating devices, i.e. to replace this:
https://github.com/TASEmulators/BizHawk/blob/c0056123cd198ba51408a2087d57c131f95e89a3/src/BizHawk.Bizware.DirectX/XAudio2SoundOutput.cs#L27
I also noticed SharpAudio
in its description which encapsulates both XAudio2 and OpenAL, but that library is .NET Standard 2.1+ (see feliwir/SharpAudio#44).
edit: and adopted Vortice recently, so the latest beta on NuGet doesn't include it
edit edit: forked it, this branch is 2 tiny commits on top of the latest beta (not on NuGet) which add .NET Standard 2.0 support, and this branch has a Nix expression which should be able to build any future beta/release w/ the same patches.
edit 2023-04: Vortice.Win32
is not suitable as a replacement for the external methods in BizHawk.Common/Win32
.
I believe that's because Vortice uses XAudio2 2.9/2.8 whereas SlimDX uses XAudio2 2.7. Microsoft stated in the changes for XAudio2 2.8:
Device enumeration functionality has been removed from XAudio2; the GetDeviceDetails and GetDeviceCount functions have been removed from the IXAudio2 interface. Apps that want to render to other audio devices on the system must pass a device identifier string to CreateMasteringVoice instead of a device index. The default audio render device can still be created without enumeration.
They don't make it clear how to get the "device identifier string" but I found a discussion where someone mentions Windows.Devices.Enumeration.DeviceInformation.FindAllAsync. I didn't want to mess with it anymore after that 😋.
I think Direct3D9 might be the bigger problem though - I did start trying to convert some of that code out of curiosity, and although I'm not very familiar with Direct3D, I'm pretty sure Vortice's implementation of D3D9 is incomplete. My impression is that the author would be unwilling to fix that himself, although would accept pull requests if someone wanted to finish the missing features.
I have a feeling we stumbled onto the same social.MSDN thread (login required)... edit: found another C++ code snippet that doesn't require login
We won't have any problems w/ D3D9 if we switch to its D3D12 backend instead. (And if we use Vortice via Veldrid then Vulkan is also available. Verily.)
This seems a good place to put this—.NET Framework 4.8.1 is real and available now, but only for Win10+, and it only includes the promised accessibility changes, not .NET Standard 2.1 support or anything "new". I don't think we should upgrade, and I'm not even going to bother checking if Mono runtime supports it.
FlatBuffers.Core
? Unknown (Emulation.Cores
). IDK, natt hacked this in for Nyma. Presumably serialisation.
Seems like since https://github.com/TASEmulators/BizHawk/commit/b56e229a45446fa62026ab16aeab201b26104bf4 our copy targets NET Standard 2.0, so shouldn't actually be any issues here with that I think.
Related google/flatbuffers#4162
Seems that this too has moved along (official NuGet available now, albeit still beta), probably not really needed for this issue though?
Saw that. I will be switching to NuGet once it's stable, even if it's not strictly necessary.
@nattthebear I've pushed branch update-flatbuffers
to do that ^ but I believe this change means that the C++ flatbuffers.h
needs updating also.
https://github.com/TASEmulators/BizHawk/blob/e87d3065718719b2abd5895886dceeb91d7f7346/waterbox/nyma/NymaTypes_generated.h#L11-L14
Seems reasonable. You should just be able to delete the contents of waterbox/nyma/common/flatbuffers/*
and replace with a new version of flatbuffers, and then rebuild all the nyma cores.
I still can't do that on my machine. With nattthebear/musl#3 I get through most of it until:
$> ./make-all-released-cores.sh
cxx mednafen/src/error.cpp
cxx mednafen/src/VirtualFS.cpp
cxx mednafen/src/FileStream.cpp
cxx mednafen/src/MemoryStream.cpp
cxx mednafen/src/Stream.cpp
cxx mednafen/src/file.cpp
In file included from mednafen/src/mednafen.h:4,
from mednafen/src/error.cpp:22:
mednafen/src/types.h:58:10: fatal error: cmath: No such file or directory
58 | #include <cmath>
| ^~~~~~~
In file included from mednafen/src/mednafen.h:4,
from mednafen/src/VirtualFS.cpp:33:
mednafen/src/types.h:58:10: fatal error: cmath: No such file or directory
58 | #include <cmath>
| ^~~~~~~
compilation terminated.
[...]
I've force pushed the branch for someone else to check. I only overwrote the flatbuffers headers that were present, assuming the rest were unused. Added the rest to resolve below problem. Can those come from a submodule instead?
I tried building them on your branch myself and ran into a different error, with "flatbuffers/array.h" missing. Seems there's also a lot of other header files missing too.
The missing <cmath>
error anyways I think is just something went wrong for you when you built the llvm/libcxx part (it should normally reside in sysroot/include/c++/v1/cmath
Pushed https://github.com/TASEmulators/BizHawk/commit/04fcf59afe75e7b4e938ea1ad0f10ab5f467529a and 3 prior commits.
Seems I got the issue with the System.Windows.Forms
thing.
BizHawk/src/BizHawk.Client.EmuHawk/MainForm.cs(3434,39): error CS1503: Argument 2: cannot convert from 'BizHawk.Client.EmuHawk.FolderBrowserEx' to 'System.Windows.Forms.CommonDialog'
My current version of .NET on Linux is 7.0, SDK Version: 7.0.101, is there no work around for this to make it work? Can we just skip the System.Windows.Forms.CommonDialog
and make it use the version in 7.0 and build anyway?
The error is something wrong from the most recent commit (affects Windows too). Just go back to the previous commit or wait until it's fixed. This is fixed.
Something to note, OpenTK.Input 4 is actually netstandard 2.0 (the rest of OpenTK is not), so us using it for host input would be (mostly) a non-blocker (host graphics/sound is obviously still an issue).
There was already an option for input on Linux, evdev-sharp
, which would have involved building from source. But while we're on the subject, allow me to quote myself:
OpenTK 4 and 5 are .NET Core only, though IIRC OTK 4 was .NET Standard until late in beta, maybe it can still work just by adding the target and building from source?
Regardless, using OpenTK.Input means no building from source, the latest nuget package is already netstandard2.0 compatible.
If we want another input backend, something to look into with Silk.NET is its SDL2 Input bindings, which may be nice in any case due to SDL2 supporting a ton of different controllers.
Doing some work on the input, it seems more complicated when I thought. OpenTK.Input in 4.x is not the same as OpenTK.Input in 3.x, as it is supposed to be some "Hid/Controller" code (in reality, there's barely anything in there and looks to be just incomplete all in all). The old OpenTK input code was moved to OpenTK.Windowing.GraphicsLibraryFramework, and has a significantly different API. Differently enough that I'm not sure we can use it anymore. The main issue is simply that it requires that an OpenTK NativeWindow
be created, and that window be focused, in order for it to obtain any input. This obviously is a bit not good, as obviously the MainForm is what we'd want focused (and since it has to be an OpenTK NativeWindow, we can't give it the MainForm anyways), and we would lose the ability to have input without focus like with OpenTK 3.x (which is probably important too for the context of controller dialog, where that might not have MainForm neccessarily focused in this context? maybe not, not sure).
(sidenote: OpenTK 4.x internally just uses GLFW to handle getting input in a platform agnostic way, while OpenTK 3.x just used its own platform abstractions to native APIs)
SDL2 similarly has this issue, but it does have an option to create a window from an existing window using a native window handle (which is handed over in the first place by our input API). Even then however, that doesn't work entirely, at least on Windows (maybe on Linux too, dunno), as for whatever reason as WM_KEYDOWN/WM_KEYUP just never arrive, despite the MainForm clearly having focus, so keyboard input just doesn't work at all.
Not entirely sure what'd be a good solution here. We could use evdev-sharp
as suggested, but since that's just evdev that will only work for Linux, and not any future macOS/whatever port and would mean Windows users are stuck with DirectInput+XInput (which is fine mostly, but it is limiting and options like SDL2 would provide more options in this respect). (Not that this doesn't mean evdev-sharp
shouldn't be a future option, but a catch-all works on all OSes would be beneficial for us to have first).
a catch-all works on all OSes would be beneficial for us to have first
I'm increasingly souring on this position. macOS all but precludes that anyway (as does Windows UWP I believe).
Continuing to look and it really just seems like we will forced to just write OS specific code for each platform for keyboard input. We can have SDL2 at least maybe as a nice catch all for gamepads (doesn't have dumb focused window requirement), but that's it. At least it seems possible across all platforms.
I currently envision this for input:
Windows: DirectInput+XInput, DirectInput+SDL2 Linux: evdev, evdev+SDL2 macOS: Quartz+SDL2
On that, this isn't perfect. We may need to change the keyboard from using DirectInput, for UWP support, as UWP does not support DirectInput, the only alternative here seems to be using the RAWINPUT API (as a bonus, we can just make DirectInput back to no longer a hard dependency this way, as the removal of OpenTK would implicitly do for Windows users). Linux doesn't necessarily have to use evdev for keyboards, X11 and libinput can work, although evdev probably would be better as a nice catch all for Linux (and BSDs?). macOS lol find someone who cares enough for some OS specific macOS code, otherwise that would stay with SDL2 for gamepad support (and I'll write up the keyboard support here).
On another note here, I don't like how the main form handle is passed on to input APIs, especially with our move to a new gui backend not making that a possibly not particularly possible idea (it is predicated on able to obtain that handle and being able to hook onto the window proc for reiniting gamepads for directinput), it would function the same if it created a hidden window internally (nb: also this would be needed for RAWINPUT use, as it responds to WM_INPUT messages), and that would be easy enough to message pump that new native window on some new thread...
EDIT: RAWINPUT handler is done and used with the SDL2 backend (also DirectInput actually wraps around RAWINPUT internally anyways, lol)
Possibly not the right thread for this: https://github.com/meziantou/Meziantou.Polyfill is similar to PolySharp, but the list of backports is different. Most relevant is that it includes some extension methods. (PolySharp seems to eschew methods added to existing types—which is fair enough, since the call must be to a nonstandard type when polyfilled i.e. not binary-compatible. But we needn't care about that for our internal libraries, only whether the combined app works.) Naively adding the dependency in results in a build failure because both trigger for the same polyfills (I'm pretty sure they run in series and the second could detect the first...), but both are also configurable. If it can reduce the amount of backports we have to maintain then I say it's worth a try.
If it wasn't obvious, these prevent us from moving to .NET 8 (see #1415).
FlatBuffers.Core
GongShell
Microsoft.VisualBasic
Newtonsoft.Json
We also have the option of replacingNewtonsoft.Json
withSystem.Text.Json
at any time.Newtonsoft.Json
, the more I think we should replace it as soon as possible. See docs supplement.System.Text.Json
uses Source Generators instead of reflection.NLua
OpenTK
replaced in dev builds from https://github.com/TASEmulators/BizHawk/commit/78f5e755343d279f6c05821751f14cea021b7672OpenTK.GLControl
replaced in dev builds from https://github.com/TASEmulators/BizHawk/commit/78f5e755343d279f6c05821751f14cea021b7672SlimDX
replaced in dev builds from https://github.com/TASEmulators/BizHawk/commit/78f5e755343d279f6c05821751f14cea021b7672System.Data.SQLite
replaced in dev builds from https://github.com/TASEmulators/BizHawk/commit/bb96825c60fbf6900bf5fd0ad20a6ea6360acfeeSystem.Drawing.Common
partially replaced in dev builds from https://github.com/TASEmulators/BizHawk/commit/7b644f771e4659a648de279fe56f61d60241ab8a<!---->—some more usages may be replaceable withMagick.NET
, otherwise seeSystem.Windows.Forms
System.Web
System.Windows.Forms
Vortice.Mathematics
dep and re-adds the target, and separately a fork of that library which copy-pastes a bunch of code from the BCL in order to re-add the target. Presumably the maintainer isn't interested in either approach.re: adopting existing libraries to replace our own code, Yoshi has some notes here. If there's enough interest in that you're welcome to open a tracking issue on GitHub instead.