search

Yellow-Dog-Man / Resonite-Issues

Issue repository for Resonite.
https://resonite.com
141 stars 2 forks source link

Official headless ARM support #2555

Open jae1911 opened 4 months ago

jae1911 commented 4 months ago

Is your feature request related to a problem? Please describe.

Currently, it is not possible to run the headless on ARM CPUs without a bit of tinkering.

The architecture is also officially unsupported.

Describe the solution you'd like

As ARM-based servers are becoming increasingly popular with cloud providers (AWS, Oracle, GCP, Azure, Hetzner), it would be nice to have official support for the ARM architecture to host headlesses.

The Oracle Cloud Free Tier is also a popular way for people to get a server to host anything they want, including headlesses. Adding an official ARM build would make it more accessible and easier to host headlesses in general.

Describe alternatives you've considered

Continue with the tinkering.

Additional Context

See Discord thread at: https://discord.com/channels/1040316820650991766/1158936045186973726

Requesters

U-j4 | j4.lc (Discord) @gameboycjp is also very active on this subject on the Discord

Frooxius commented 4 months ago

The main thing we need for this are ARM versions of the core libraries we use. We do have repositories for these open, so it's possible for community to contribute CI/CD scripts that would get those versions built which would help this.

We can put a list together for which ones would be needed.

jae1911 commented 4 months ago

We can put a list together for which ones would be needed.

That would be helpful!

gameboycjp commented 4 months ago

FreeImage, FreeType, and Opus are the minimum for the engine and world to start, but anything else that relies on unmanaged binaries like assimp, brotli, and msdfgen also do not work and throw errors when called.

gameboycjp commented 4 months ago

There's also Steamworks.NET targeting x86/64 exclusively and causing an error when reflection is used on modern .NET. Setting it to build for AnyCPU fixes the problem.

ticpu commented 4 months ago

What about making Steamworks optional for the headless? It isn't required for anything useful that I know of when you setup your port forwards and all.

Edit: Like j4 says, it is very useful for Oracle hosting, it is a very performant alternative to most free x86 offerings.

gameboycjp commented 4 months ago

I'm not sure that's necessary. Once Steamworks.NET is built for AnyCPU, the headless works as it does on mono. I think I'll put together a simple PR, I just need to figure out the best approach for the change.

Frooxius commented 4 months ago

I'm a bit puzzled on how would Steamworks work on ARM? It's a wrapper for the native Steam library and that one is for x64 as far as I'm aware.

Does Steam offer ARM version of it?

gameboycjp commented 4 months ago

Does Steam offer ARM version of it?

It doesn't, but the wrapper needs to be able to be reflected against during engine initialization. I can provide more info here or in #2545, but fixing that is a prerequisite for getting the .NET 8 headless to load on ARM.

ticpu commented 4 months ago

I'm not sure why my comment hasn't been considered, on the mono version, it currently fails to load and continues loading the worlds on my ARM instance. What is the use of Steamworks on the headless in the first place? Can it just be made as an optional dependency?

Logs for my current ARM instance:

17:06:38.039            Steam
17:06:38.075    Exception initializing Steamworks.NET!
System.DllNotFoundException: steam_api assembly:<unknown assembly> type:<unknown type> member:(null)
<stack trace continues...>
at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00000] in <12b418a7818c4ca0893feeaaf67f1e7f>:0
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <12b418a7818c4ca0893feeaaf67f1e7f>:0
17:06:38.090    Supported network protocols: lnl-nat, lnl
17:06:38.092    Initializing SkyFrost Interface...
17:06:38.135    Proxy configuration for skyfrost not found or failed to initialize.
gameboycjp commented 4 months ago

Judging from the logs, it seems to fail to get the types from FrooxEngine, because Steamworks.NET fails to load with reflection. It's possible that the types in Steamworks.NET are used in FE. The previous behavior of it just not using Steamworks for networking when it fails to load seems fine to me.

Frooxius commented 4 months ago

Steamworks is used for Steam Networking Sockets protocol. Without it, this protocol won't be available.

It's not going to make the rest not work, so it should work fine regardless. Ideally it should just fail to load and continue without it, so fixing up that would be the first step for that.

gameboycjp commented 4 months ago

Is it something you would like to work around on the type scanning side, or on the Steamworks.NET side? If on the latter, I think I'll rework my current dirty patch into something cleaner and PR it.

charlie-sans commented 2 months ago

hey! how's progress on this issue going. just wanting to see if there is anything that i can look into for assistance for yall

ProbablePrime commented 2 months ago

hey! how's progress on this issue going. just wanting to see if there is anything that i can look into for assistance for yall

No one is actively working on this right now.

You can see what we need here: https://github.com/Yellow-Dog-Man/Resonite-Issues/issues/2555#issuecomment-2225930749

gameboycjp commented 1 month ago

The main thing we need for this are ARM versions of the core libraries we use. We do have repositories for these open, so it's possible for community to contribute CI/CD scripts that would get those versions built which would help this.

I've been working on these, and I finally got around to asking where some of the ones that weren't forked by y'all were in Primes office hour today. Where would I contribute those CI scripts to, for the libraries that aren't forked?

Frooxius commented 1 month ago

The main thing we need for this are ARM versions of the core libraries we use. We do have repositories for these open, so it's possible for community to contribute CI/CD scripts that would get those versions built which would help this.

I've been working on these, and I finally got around to asking where some of the ones that weren't forked by y'all were in Primes office hour today. Where would I contribute those CI scripts to, for the libraries that aren't forked?

Make a request for us to put up forks for any that are missing and we'll sort them out! We just need a list of ones that are missing to look through.

gameboycjp commented 1 month ago

Opus, Brotli, and FreeType are what I'm currently looking at. I think there might be more, though... I'll look through the list of native binaries included with the current x64 headless and see if I can find more missing repos at some point.

ticpu commented 1 month ago

Please also consider a single repo than uses submodules where a fork isn't necessary as to keep things in line with upstream as much as possible. I don't see the day where libopus will have to be customized for Resonite.

I've been using my system's own library for most of the dependencies for the headless, even the forked one for some, and it just seemed to work. And it is even better as the distro keep them updated for security versus a fork.

Frooxius commented 1 month ago

Even if we might not require modifications, we usually fork so we can verify the changes coming into Resonite and make sure they don't break things (which has happened a few times by just updating a dependency).

gameboycjp commented 1 month ago

Two I missed mentioning are Compressonator and PDFium.

Frooxius commented 1 month ago

Both of these we are just using the prebuilt ones.

Compressonator is not supported on Linux right now, we'd need to setup pipeline to build shared libraries for those.