Running X-less

[vanfanel]

Hi there,

I have found about this project when someone commented it on the Scummvm forums, and I was just awed by what's being done here, as Ambermoon is a LEGEND for me.

Now, I was thinking about trying this on a Raspberry Pi, which ideally would mean that the game should run outside X11 somehow, as that's where the Pi performs better.

So, what would be the best (if any) approximation to the problem? I don't know anything about the technology this engine uses (is monogame the way to go on GNU/Linux?), but whatever virtual machine this engine uses, it should support KMSDRM/GLES directly, or SDL2 which in turn has KMSDRM support. Any thoughts?

[Pyrdacor]

Hi. The game uses Silk.net as a graphic and window backend which itself uses glfw. But the most recent version should also support SDL so there might be a chance. I would need you as a tester to make this possible.

I could setup the build pipeline etc and update the Silk.net dependency. I can't promise that it will work though.

[Pyrdacor]

About your question of how C# runs on GNU/Linux. There was the .NET Framework which was only for Windows back then. Then Mono was born which rewrote the framework on Linux and became even better. Then Microsoft gave up and released the source. NET Core was born which was then cross platform but not as feature-rich. Modern projects (including this here) use NET Core to be cross platform. So you don't need Mono anymore. You might have to install NET Core on your distro but that's all.

[vanfanel]

@Pyrdacor Great! I see silk.net supports SDL now acording to their github repository here: https://github.com/dotnet/Silk.NET ...so this should (in theory) work. I understand the Ambermoon.net project will somehow download and build Silk.net itself so you have to update it?

[Pyrdacor]

@vanfanel Right Ambermoon.net references Silk.net via a so called nuget package. It references a specific version so I have to update that version and rebuild. Moreover I might have to adjust some code as the major version changed. Then we could test. Will do so next week.

[vanfanel]

@Pyrdacor Ok! Will test when that happens :)

[vanfanel]

@Pyrdacor I understand this is the runtime I need, right? (I am running aarch64 GNU/Linux)

https://dotnet.microsoft.com/download/dotnet/thank-you/runtime-3.1.14-linux-arm64-binaries

..and no SDK needed, just the runtime. Is that correct?

[Pyrdacor]

Yes. It's exactly what you need.

If you want you can also try to build it yourself. Then you will need the SDK instead of the runtime. You can use the dotnet command line tools to build the solution. The Silk.net versions can be found inside the csproj files.

But of course you can wait until next week and I will take care of it. ;)

[vanfanel]

@Pyrdacor Ah yes, just tried to build and got the message saying that I need the SDK. I will wait. My love for Microsoft isn't exactly great, so the less SW from then I need to use, the better :)

[Pyrdacor]

Who would blame you for that? :D I am not a big fan either but C# is really nice and Visual Studio as well.

[vanfanel]

@Pyrdacor In the end I went and built it on my X86_64 laptop running Lubuntu. So, it seems that ./dotnet build ~/src/Ambermoon.net/Ambermoon.net.sln succeeded, at least. In your instructions on the other issue, you say that, at this point, it's possible to run the project with dotnet run, but what should I pass to dotnet run?

[Pyrdacor]

I think you have to give the project path. Something like ./dotnet run -p ~/src/Ambermoon.net/Ambermoon.net/Ambermoon.net.csproj.

Or just cd into the Ambermoon.net subfolder and hit dotnet run without parameters there.

[Pyrdacor]

Alternatively you can go to the output path of the Ambermoon.net project (bin/...) and call dotnet Ambermoon.net.dll there. It might by a Ambermoon.net.so in your case maybe.

[vanfanel]

@Pyrdacor I had to copy over the dotnet SDK to the Ambermoon.net dir, and after that I tried but failed to run:

manuel@hp15db0:~/src/Ambermoon.net$ ./dotnet run -p Ambermoon.net/Ambermoon.net.csproj
UI/Layout.cs(1042,18): warning CS8321: The local function 'ToggleMusic' is declared but never used [/home/manuel/src/Ambermoon.net/Ambermoon.Core/Ambermoon.Core.csproj]                                                                                                    
UI/Layout.cs(1049,18): warning CS8321: The local function 'ToggleFastBattleMode' is declared but never used [/home/manuel/src/Ambermoon.net/Ambermoon.Core/Ambermoon.Core.csproj]                                                                                           
Game.cs(218,23): warning CS0414: The field 'Game.legacyMode' is assigned but its value is never used [/home/manuel/src/Ambermoon.net/Ambermoon.Core/Ambermoon.Core.csproj]                                                                                                  
Unable to find any dictionary file.                                                                                                   
   at Ambermoon.Data.Legacy.GameData.Load(Func`2 fileLoader, Func`2 diskLoader, Func`2 fileExistChecker) in /home/manuel/src/Ambermoon.net/Ambermoon.Data.Legacy/GameData.cs:line 329
   at Ambermoon.Data.Legacy.GameData.Load(String folderPath) in /home/manuel/src/Ambermoon.net/Ambermoon.Data.Legacy/GameData.cs:line 205
   at Ambermoon.GameWindow.ShowVersionSelector(Action`3 selectHandler) in /home/manuel/src/Ambermoon.net/Ambermoon.net/GameWindow.cs:line 356
   at Ambermoon.GameWindow.Window_Load() in /home/manuel/src/Ambermoon.net/Ambermoon.net/GameWindow.cs:line 501
   at Silk.NET.Windowing.Desktop.GlfwWindow.Initialize()
   at Silk.NET.Windowing.Common.WindowExtensions.Run(IView view)
   at Ambermoon.GameWindow.Run(Configuration configuration) in /home/manuel/src/Ambermoon.net/Ambermoon.net/GameWindow.cs:line 617
   at Ambermoon.Program.Main() in /home/manuel/src/Ambermoon.net/Ambermoon.net/Program.cs:line 54

[vanfanel]

Also tried to run the binary, which seems to be a .dll on GNU/Linux too:

manuel@hp15db0:~/src/Ambermoon.net$ ./dotnet run Ambermoon.net/bin/Debug/netcoreapp3.1/Ambermoon.net.dll 
Couldn't find a project to run. Ensure a project exists in /home/manuel/src/Ambermoon.net, or pass the path to the project using --project. 

[Pyrdacor]

The first worked. The dev environment needs the original game data files. Or you have to use dotnet publish and merge the file versions.dat to it. You can see how in the publish.ps1.

But it's easier to get the original game data and put it next to the dll. ADF files or extracted folders.

[Pyrdacor]

You can try the following:

dotnet publish -c Release "./Ambermoon.net/Ambermoon.net.csproj" -p:PublishSingleFile=true -r linux-arm64 --no-restore --no-self-contained
dotnet publish -c Release "./Ambermoon.ConcatFiles/Ambermoon.ConcatFiles.csproj" -r linux-arm64 --no-restore
./Ambermoon.ConcatFiles/bin/Any CPU/Release/netcoreapp3.1/linux-arm64/publish/Ambermoon.ConcatFiles ./versions.dat "./Ambermoon.net/bin/Any CPU/Release/netcoreapp3.1/linux-arm64/publish/Ambermoon.net"

Then you might start the resulting assembly without any game data.

The part "Any CPU" might be omitted in the paths dependent on environment.

[Pyrdacor]

Hm I spoke too soon. The CI that I use can't build arm yet. So I guess you have to do it on your own for now. But I will add some scripts and docs to help you.

At least I added the arm64 config to the projects. So you should git pull again to make my code examples with dotnet publish above work.

[vanfanel]

I am trying to run it on X86_64 for now, Pi4 (aarch64) will come later. Well, patience...

I am trying with ./dotnet run -p Ambermoon.net/Ambermoon.net.csproj since you said that worked. Where exactly should the original game datafiles go? You said to put it next to the dll, but as you saw, running the DLL via ./dotnet run Ambermoon.net/bin/Debug/netcoreapp3.1/Ambermoon.net.dll failed.

EDIT: Ah, this worked to run the DLL:

./dotnet Ambermoon.net/bin/Debug/netcoreapp3.1/Ambermoon.net.dll

Now I suppose I have to put the game data in Ambermoon.net/bin/Debug/netcoreapp3.1/, right?

EDIT2: Oh, I put all files there, and I get this now:

manuel@hp15db0:~/src/Ambermoon.net$ ./dotnet Ambermoon.net/bin/Debug/netcoreapp3.1/Ambermoon.net.dll 
Unable to find travel graphics.
   at Ambermoon.Data.Legacy.GameData.LoadTravelGraphics() in /home/manuel/src/Ambermoon.net/Ambermoon.Data.Legacy/GameData.cs:line 378
   at Ambermoon.Data.Legacy.GameData.Load(Func`2 fileLoader, Func`2 diskLoader, Func`2 fileExistChecker) in /home/manuel/src/Ambermoon.net/Ambermoon.Data.Legacy/GameData.cs:line 329
   at Ambermoon.Data.Legacy.GameData.Load(String folderPath) in /home/manuel/src/Ambermoon.net/Ambermoon.Data.Legacy/GameData.cs:line 205
   at Ambermoon.GameWindow.ShowVersionSelector(Action`3 selectHandler) in /home/manuel/src/Ambermoon.net/Ambermoon.net/GameWindow.cs:line 356
   at Ambermoon.GameWindow.Window_Load() in /home/manuel/src/Ambermoon.net/Ambermoon.net/GameWindow.cs:line 501
   at Silk.NET.Windowing.Desktop.GlfwWindow.Initialize()
   at Silk.NET.Windowing.Common.WindowExtensions.Run(IView view)
   at Ambermoon.GameWindow.Run(Configuration configuration) in /home/manuel/src/Ambermoon.net/Ambermoon.net/GameWindow.cs:line 617
   at Ambermoon.Program.Main() in /home/manuel/src/Ambermoon.net/Ambermoon.net/Program.cs:line 54

[Pyrdacor]

Do you use the ADF disk files or the extracted folder? If extracted you should put the files directly into the folder with the DLL (not the Amberfiles folder). Ensure the file "Travel_gfx.amb" is next to the DLL and the other files as well.

You can also use the ambermoon.cfg to specify the game data path. It should be generated in the same folder.

[vanfanel]

@Pyrdacor There's something amiss with the "Travel_gfx.amb" file. I have searched in all the Ambermoon datafiles I could find, including the whdownload.com available versions, and the ones available in EAB's fileserver that I can decompress, and the file isn't anywhere to be found...

[Pyrdacor]

Just try the ADF versions. You can download from my other github repo: https://github.com/Pyrdacor/Ambermoon/tree/master/Disks.

There are english or german versions. Put the ADF files next to the DLL. Should work.

I wonder why the file is missing. It's a key part of the game.

[vanfanel]

@Pyrdacor Those games in your repo (which seem to be from the same version found on the Thalion Shrine, right?) are in DMS format. So I tried to copy them as they come in the ZIP, and data couldn't be found. Tried to rename them from DMS to ADF, and data can't be found either.

[Pyrdacor]

Yeah you are right, sorry. Only the german 1.01 version is ADF. You could test with it for now. There are tools to convert DMS to ADF I guess. Maybe only for the Amiga. So if you have an emulator at hand you could convert it.

I will provide ADF and extracted versions in my repo of all game versions this week. But not today as it is my birthday. ;)

[vanfanel]

@Pyrdacor Happy birthday then! Mine was two days ago, just turned 40. Enjoy your day and stay away from computer, just enjoy :)

[Pyrdacor]

Congratulations. 40 is not far for me too. ;) Thanks. Yeah today is computer-free. I will help you the next days.

[Pyrdacor]

You can find the ADF and extracted english versions now at: https://github.com/Pyrdacor/Ambermoon/tree/master/Disks/English

I also added tar.gz for our Linux fellas. :)

[vanfanel]

@Pyrdacor Thanks! Everything seems to be working on GNU/Linux X86_64 X11 (except for music, but I guess music is not yet implemented).

Next destination: X11-less.

I have been experimenting with FNA these days and, since FNA games use SDL2, they work X11-less. So, could Ambermoon.net use FNA+SDL2 or am I asking about a too different technology there? (I take dotnet and FNA both use C#, so...).

EDIT: I have found out how to build a RELEASE version, with ./dotnet build ~/src/Ambermoon.net/Ambermoon.net.sln --configuration Release but RELEASE builds don't go beyong the tittle screen.

[Pyrdacor]

I added a build script (arm64.sh) to the root dir. Executing it there should build for arm64.

It should merge the built-in game data into the assembly as well so you should be able to play it.

[vanfanel]

Great! Should it run without X11 then? Any special steps to do so?

[Pyrdacor]

It should at least run on arm64. The SDL support is not there. I will have a look at it soon.

[vanfanel]

@Pyrdacor The problem with no SDL2 support is that I can't test on aarch64 as I don't have an aar64 system with desktop installed... So I will have to wait for SDL2 support to test on aarch64.

[Pyrdacor]

I will come back to SDL support after the next release. Switching the major version of Silk.net might have some breaking changes so I will need some time for it. When adding music I will have to add or rethink dependencies regardless as cross platform sound support is a huge thing. So I guess this will be the right time to switch to the most recent Silk version and therefore hopefully add SDL2 support.

[Pyrdacor]

@vanfanel It's been a while. I noticed that the SDL support should be already there now. Can you try to run the recent version (0.9.3beta) on aarch64? You can use the mentioned script arm64.sh. Let me know how it works.

Note: There might be some input issues with SDL (mouse and keyboard) which I am currently investigating so if this happens it is a known issue.

[Pyrdacor]

I talked to the maintainers of Silk.NET about SDL support. They only added it for mobile support. And for the next major release it might be removed all together.

Is there no way to use glfw? According to the web, glfw should run just fine on pi. If not I guess the problem could be OpenGL. You'll need at least OpenGL 3 support to run the game.

[vanfanel]

@Pyrdacor Sorry I didn't answer earlier. It seems that github isn't sending me mails about conversation answers...

Raspberry Pi 4 is limited to OpenGL 2.1, it doesn't (and I think won't) support OpenGL 3.x. So... no way it will work I guess.

[Pyrdacor]

@vanfanel I'm sorry about this but OpenGL 2.1 is 15 years old. It would be a lot of effort to redesign the engine so that it would work with 2.1 I guess.

[vanfanel]

@Pyrdacor What about GLES3? The Pi4 supports OpenGL ES up to version 3.2.

[Pyrdacor]

I guess this should work. I will try to allow it. And then maybe you can test it.

[vanfanel]

@Pyrdacor Great! I will be glad to test it on the Pi if GLES 3.2 compatibility is added! :)

[Pyrdacor]

@vanfanel I am not sure if building went right but can you test this version please? https://github.com/Pyrdacor/Ambermoon.net/releases/download/TestRelease/Ambermoon.net-Pi4-Standalone.tar.gz

[Pyrdacor]

I eased the requirements to GLES 3.0 as it should be enough.

[Pyrdacor]

@vanfanel Can you test the pi4 version?

[a1exh]

I have a Pi400 I could try it with later. But I don't fully understand the test requirements? PiOS without a desktop?

[Pyrdacor]

I have a Pi400 I could try it with later. But I don't fully understand the test requirements? PiOS without a desktop?

I am not quiet sure. I guess if it runs on Pi4 all is good.

[vanfanel]

@Pyrdacor Sorry I couldn't test... I will do so tomorrow I hope!! I am not at home so I don't have my Pi4 here. @a1exh Hi! I know you from the EAB forums! :) If the game runs on the Pi4, it will run on the 400.

[vanfanel]

@Pyrdacor I could finally test it on the Pi4! I had very high hopes because I have seen it's an aarch64 executable, but upon execution I get this message:

pi@raspberrypi:~/src $ ./Ambermoon.net 
Couldn't find a suitable window platform. https://docs.ultz.co.uk/silk.net/windowing/troubleshooting.html
   at Silk.NET.Windowing.Window.GetView(Nullable`1 options)
   at Ambermoon.GameWindow.Run(Configuration configuration)
   at Ambermoon.Program.Main()

[Pyrdacor]

Can you try to install glfw manually? I saw that there are versions for arm64 as well and some that use wayland/mesa instead of X11. Maybe you can install the lib and it just works. Glfw should at least run on Pi4 somehow as a few people already use it.

It is really hard to test this for me. I tried to cross compile glfw on Ubuntu for Raspberry this morning but without luck. :/ But as I said there seem to be library versions already that should run on Pi.

[vanfanel]

@Pyrdacor I am not using X11/Wayland on the Pi4 (everything in SDL2 and libretro run on KMS/DRM, no graphics server is needed), so GLFW would have to support creating a KMS/DRM context. That doesn't seem to be the case.

However, I have opened an issue in case the GLFW people can confirm:

https://github.com/glfw/glfw/issues/1938

Wouldn't it be possible for you to use SDL2 for rendering instead of using OpenGL directly? (You can create an OpenGL window in SDL2 with a single function call, at least in C/C++)

[Pyrdacor]

Wouldn't it be possible for you to use SDL2 for rendering instead of using OpenGL directly? (You can create an OpenGL window in SDL2 with a single function call, at least in C/C++)

To make it short: no. The whole engine is selfmade and based on plain OpenGL calls. To reimplement this all in SDL is a huge project on its own and I won't have the time for it.

And the problem is not only rendering but the window creation which is a pain in the *** most of the times. At least in a cross platform way.

I fear I can't support any window system on any platform. Afaik wayland and/or X11 are both available on Raspberry Pi and one of them should be the default. So supporting these should be enough for now.

But glfw is a good choice for cross platform imo so maybe we should ask them to implement KMS/DRM.

[vanfanel]

@Pyrdacor There's a small detail that I would like to comment: if the engine uses direct OpenGL calls, that's not a problem with SDL2. You create the OpenGL window (a single SDL_CreateWindow() call) and then you can start sending OpenGL commands, the same OpenGL commands you currently use. And SDL2 supports KMS/DRM for you, you don't need to even know about it! :)

Not trying to convince you, just making it clear how simple it would be with SDL2. SDL2 gives you support for all major windowing systems... for free. You don't need to know anything about them, just create an SDL2 window, and SDL2 takes care of everything under the hood.