search

Neos-Metaverse / NeosPublic

A public issue/wiki only repository for the NeosVR project
196 stars 9 forks source link

ARM / Raspberry Pi Linux support (headless only) #2766

Open Frooxius opened 3 years ago

Frooxius commented 3 years ago

Is your feature request related to a problem? Please describe.

Currently the native Linux support only supports the x64 CPU architecture, so running it on Raspberry Pi isn't possible without some form of emulation.

Relevant issues

No response

Describe the solution you'd like

Compile the native binaries for ARM/Raspberry Pi Linux, in order to allow running headless on those devices. Currently graphical client isn't likely to be supported with this, since Unity doesn't support this, but headless could be interesting option.

Additionally we'd have to make this part of some distribution process. Steam currently doesn't handle Raspberry Pi natively, so we'd have to include the binaries in a separate folder or wait until we have our own update service to handle this properly.

Describe alternatives you've considered

N/A

Additional context

This is something we get asked on occasional basis, so since I've been splitting up the Linux issues from the megathread (https://github.com/Neos-Metaverse/NeosPublic/issues/149), I wanted to log this as well.

This is currently mainly done to gauge interest in this. Without interest we aren't likely to prioritize this anytime soon, unless we're doing some fun project and would like to pick it up.

However this is also an issue where community help would be welcome. Most of our native dependencies are open source projects available on our GitHub: https://github.com/Neos-Metaverse

If anyone is interested in helping get this implemented, it would require setting up build scripts for the Azure Pipelines for those projects. I can provide more guidance in case anyone's interested in taking this up.

RussoNC commented 3 years ago

I would love to work on this ! (Frenx on discord and in/game) I am not sure how to proceed thought, I would like to first do a test compiling it myself in ARM but I'm not sure where to start, should I take every dependency on the Neos repo and compile it on the raspi? or could I, for example create a docker image and run it there.

Frooxius commented 3 years ago

Hello! Thanks for your interest in this! :D

There are several native libraries that need to be compiled for Raspberry Pi in order to get Neos headless running there. At very least, these are crucial:

There's a few that are more optional (stuff should mostly work without them, with some functionality loss, like not being able to import 3D models on the headless):

We have forks for all of these, some of them contain our custom modifications. All of these need to be compiled as shared libraries (.so) so they can be dynamically linked and invoked through P/Invoke from C#.

For most of these, there should be neos-release branch. This branch contains azure-pipelines.yml, which currently compiles them for Windows (x64), Linux (x64) and Android (ARMv7). The Linux one should be a good reference for the compilation steps here.

For this to get to work properly, we'd need to add another compilation branch to the azure-pipelines.yml to produce Raspberry Pi compatible versions. I'd probably recommend trying out the compilation locally first and then running the headless with Mono, seeing if it takes the library and then adding the steps to azure-pipelines.yml so the new versions get built whenever we merge updates from upstream, otherwise the compiled binaries will get out of sync.

For it to compile for Raspberry Pi though you'll have to cross-compile though. From what I found there's a tooling for CMake that a bunch of these use that can be used and should make things hopefully simpler.

I hope this helps, let me know if you have any questions!

RussoNC commented 3 years ago

Thanks a lot for the detailed answer ! I will try to work on this ASAP. Time to give another life to my sleeping Raspy4. I will probably have more questions later on, and I'll give news if my build works first locally.

peterthethinker commented 3 years ago

as a datapoint. Pi4 is popular. the ODROID cost more but often have WAYYYYYYYY more power to them and Ram ect. >

Would the intent be to have it for any ARM modern chipset or just Pi4 ?

I ask as some of the ODroids have baked in SATA and that would be a nice way to slap a nice HUGE asset drive as neos storage .

shadowpanther commented 3 years ago

As ODroids seem to also have an ARM SoC, and if Mono runs on ODroid (a thing to check for someone that has it), just compiling the required libraries for the ARM architecture should be enough to support both RPi and ODroid.

If you have a board to experiment on, try to test whether you can compile the libraries listed by Froox above on it.

Frooxius commented 2 years ago

I don't think there's anything special on Raspberry Pi binaries, so I think they should work on any compatible ARM device with Linux (assuming it's same ARM architecture). Not 100 % sure though, I haven't tested it.

FlameSoulis commented 2 years ago

So I went ahead and tried just doing a blind run of the headless client on a new Pi4 I received. It will start okay with some confusion about the system specs, but will fail upon loading FreeImage related stuff.

ubuntu@ubuntu:~/neos$ mono Neos.exe
Initializing Neos: Beta 2021.11.10.1253
CPU: UNKNOWN, Processor Count: 4 (Physical: UNKNOWN)
Max GC Generation: 1, IsLittleEndian: True
System Memory: -1.00 B
GPU: UNKNOWN, VRAM: -1.00 B, POT Byte Aligned: True
System.Numerics.Vectors HW accelerated: False, Vector<float>.Count: 4
HeadDevice: Headless
6.8.0.105 (Debian 6.8.0.105+dfsg-2 Wed Feb 26 23:41:24 UTC 2020)
Supported Texture Formats: Unknown, Alpha8, RGB24, ARGB32, RGBA32, BGRA32, RGBAHalf, ARGBHalf, RGBAFloat, ARGBFloat, BC1, BC2, BC3, BC4, BC5, BC6H, BC7, ETC2_RGB, ETC2_RGBA1, ETC2_RGBA8, ASTC_4x4, ASTC_5x5, ASTC_6x6, ASTC_8x8, ASTC_10x10, ASTC_12x12
Argument: /home/ubuntu/neos/Neos.exe
Available locales: cs, de, en, en-gb, eo, es, et, fi, fr, is, ja, ko, nl, no, pl, ru, sp, sv, tr, zh-cn, zh-tw
Loading Config.json...
Computing compatibility hash...
Compatibility Hash: eHda/ALLXXI1UIr+78qg4A==
Initializing FrooxEngine...
        Initializing Libraries
Configuring System.Net.ServicePointManager
Calling Coder<T>
Performing a dummy texture decode
Unhandled Exception:

System.TypeInitializationException: The type initializer for 'CodeX.TextureDecoder' threw an exception. ---> System.TypeInitializationException: The type initializer for 'FreeImageAPI.Plugins.PluginRepository' threw an exception. ---> System.DllNotFoundException: FreeImage assembly:<unknown assembly> type:<unknown type> member:(null)
  at (wrapper managed-to-native) FreeImageAPI.FreeImage.GetFIFCount()
  at FreeImageAPI.Plugins.PluginRepository..cctor () [0x00000] in <4c4d3d8df61a4031ace2678adeb3a87f>:0
   --- End of inner exception stack trace ---
  at CodeX.TextureDecoder..cctor () [0x0001b] in <97504c9c155c41c180cc3e771d88eddb>:0
   --- End of inner exception stack trace ---
  at FrooxEngine.LibraryInitializer.Initialize (FrooxEngine.Engine engine) [0x0006b] in <5c4d7b9711784234a3fa14befde4fcf9>:0
  at FrooxEngine.EngineInitializer.InitializeFrooxEngine (FrooxEngine.Engine engine) [0x00024] in <5c4d7b9711784234a3fa14befde4fcf9>:0
  at FrooxEngine.Engine.Initialize (System.String appPath, System.String dataPath, System.String cachePath, FrooxEngine.ISystemInfo systemInfo, FrooxEngine.IInternalResource resources, System.Boolean verboseInit) [0x0099b] in <5c4d7b9711784234a3fa14befde4fcf9>:0
  at FrooxEngine.StandaloneFrooxEngineRunner.Initialize (System.String dataPath, System.String cachePath, System.Boolean verbose) [0x000c4] in <5c4d7b9711784234a3fa14befde4fcf9>:0
  at NeosHeadless.Program.Main (System.String[] args) [0x00492] in <fae53b88dc494fb2bf21b8dfbb940921>:0
  at NeosHeadless.Program.<Main> (System.String[] args) [0x0000c] in <fae53b88dc494fb2bf21b8dfbb940921>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.TypeInitializationException: The type initializer for 'CodeX.TextureDecoder' threw an exception. ---> System.TypeInitializationException: The type initializer for 'FreeImageAPI.Plugins.PluginRepository' threw an exception. ---> System.DllNotFoundException: FreeImage assembly:<unknown assembly> type:<unknown type> member:(null)
  at (wrapper managed-to-native) FreeImageAPI.FreeImage.GetFIFCount()
  at FreeImageAPI.Plugins.PluginRepository..cctor () [0x00000] in <4c4d3d8df61a4031ace2678adeb3a87f>:0
   --- End of inner exception stack trace ---
  at CodeX.TextureDecoder..cctor () [0x0001b] in <97504c9c155c41c180cc3e771d88eddb>:0
   --- End of inner exception stack trace ---
  at FrooxEngine.LibraryInitializer.Initialize (FrooxEngine.Engine engine) [0x0006b] in <5c4d7b9711784234a3fa14befde4fcf9>:0
  at FrooxEngine.EngineInitializer.InitializeFrooxEngine (FrooxEngine.Engine engine) [0x00024] in <5c4d7b9711784234a3fa14befde4fcf9>:0
  at FrooxEngine.Engine.Initialize (System.String appPath, System.String dataPath, System.String cachePath, FrooxEngine.ISystemInfo systemInfo, FrooxEngine.IInternalResource resources, System.Boolean verboseInit) [0x0099b] in <5c4d7b9711784234a3fa14befde4fcf9>:0
  at FrooxEngine.StandaloneFrooxEngineRunner.Initialize (System.String dataPath, System.String cachePath, System.Boolean verbose) [0x000c4] in <5c4d7b9711784234a3fa14befde4fcf9>:0
  at NeosHeadless.Program.Main (System.String[] args) [0x00492] in <fae53b88dc494fb2bf21b8dfbb940921>:0
  at NeosHeadless.Program.<Main> (System.String[] args) [0x0000c] in <fae53b88dc494fb2bf21b8dfbb940921>:0

After reviewing the above-mentioned important libraries, I went ahead and downloaded the forked version and attempted to compile them on the Pi natively.

I have not tried the optional libraries yet, and I'm still struggling to find a way to ensure I'm also building the .NET versions, especially for FreeImage (in which FreeImage.NET is no longer supported and there were virtually no build guides to begin with for Linux systems).

gameboycjp commented 2 years ago

I think FreeImage.NET is a wrapper? Does it need to be built against the dll it's going to wrap? Correct me if I'm wrong on these things. I'd like to contribute more but I've barely have any knowledge on this stuff, but I really want to learn.

liny-fox commented 2 years ago

As for compiling freeimage, trying with the latest armbian ubuntu 22.04, the version of c++ that gcc tries to use is too new for the code and errors out on features removed sense c++17 (deprecated in c++11), so would need to update the code or force it to build as c++14

Crunch:

This was resolved by using the default break point mentioned later by changing line 38 to the following:

#elif !defined(__GNUC__) && !defined(ANDROID)

A better fix for crunch is

#elif defined(__GNUC__) && !defined(ANDROID) && ( defined(__i386__) || defined(__x86_64__) )

as the int$3 line is specific to intel 386 and sibling arch's, so should really only be used there.

Freetype: Compiled fine when configuring it with cmake.

BlueCyro commented 2 years ago

So I am compiling for an ARM Ampere cloud VM from oracle since a lot of what people said here applies. I was able to force FreeImage to compile for c++14, but I've run into a snag. It appears to spit out a bunch of errors for VP8GetBit, VP8GetBitAlt and VP8GetSigned in FreeImage/Source/LibWebP/src/utils/bit_reader_inl_utils.h : image

The two main errors seem to be as follows:

error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token error: macro "(InsertMacroHere)" requires 3 arguments, but only 2 given

I'm looking into these but it's been inconclusive so far.

BlueCyro commented 2 years ago

So I've found the culprit. Looks like someone was naughty and tried to do macros to remove an argument in bit_reader_utils.h, which isn't supported. However, variadic macros are supported, so making a bunch of the macros for the VP8Get- functions variadic (by replacing the last variable with ... fixed it, and adding a 0 to another problematic function call in another file that tried to use less functions than it was supposed to allowed it to fully compile. Pics of the changes I made:

The variadic macros image

And the function I added a 3rd argument to image

Specifically the function shown in that second picture has code to check that 3rd parameter, so I thought it fine to just add it for the sake of laziness. image

Now the problem is that I've run sudo make install on all of them except for crunch, which I've manually moved to the /lib folder, but the headless still doesn't see them. I'm unfamiliar with how to properly link files in linux, so help would be appreciated here.

BlueCyro commented 2 years ago

So while I've managed to get FreeImage to compile, I'm getting a message upon starting a world on the headless from mono:mono: symbol lookup error: /home/ubuntu/Neos Server/NeosVR/libFreeImage.so: undefined symbol: png_init_filter_functions_neon

Talking to geenz reveals that this is some sort of issue with an SIMD instructionset or something not being enabled (which I've looked up, and it appears that it should be enabled by default on aarch64). For context I'm trying to compile on an oracle cloud VM with an Ampere ARM cpu.

The pngpriv.h file within Source/LibPNG details some of how to enable it, but I've tried setting CXXFLAGS += -DPNG_ARM_NEON_OPT=1 in Makefile.gnu in the main directory, along with trying to just set it on CFLAGS as well, but nothing I've tried so far makes even a lick of difference.

I was also told to look at the azure pipelines branch to get an idea of how compiling FreeImage is done on linux, but either I'm too inexperienced or the only thing related to azure pipelines is azure-pipelines.yml in the azure pipeline branch, and that file isn't descriptive on how to do this on linux.

FlameSoulis commented 2 years ago

Somehow, I recall someone mentioning that FreeImage was available as a package on Debian distros, but this was long ago, and I don't have access to the support channels anymore.

I haven't dabbled with this for a while, so I'll first try at least on a Pi and see if those findings help with the VMs. As mentioned in my last post, FreeImage was the roadblock for me.

BlueCyro commented 2 years ago

Somehow, I recall someone mentioning that FreeImage was available as a package on Debian distros, but this was long ago, and I don't have access to the support channels anymore.

I haven't dabbled with this for a while, so I'll first try at least on a Pi and see if those findings help with the VMs. As mentioned in my last post, FreeImage was the roadblock for me.

I've managed to compile everything correctly! FreeImage doesn't compile for the neon optimizations properly so I had to disable them (though I would've liked them to be enabled and I'll keep looking for how to enable them). And I'll be documenting the processes I went through in order to get this to work.

In the meantime, enjoy this image! image

PointerOffset commented 2 years ago

I can confirm this is working as well. Following Cyro's tweaks I was able to compile the required libraries on an M1 Mac Mini running Asahi Linux and get a headless client going. It's running now and seems to be stable.

The future of running headless on ARM is looking bright! 👍

BlueCyro commented 2 years ago

Hi everyone. This was one massive headache to pull off but I've finally done it. After going through the painstaking process of building these libraries for arm, I've forked a couple repos and made a (hopefully) easy-to-follow guide on how to build these libraries for yourself. https://github.com/RileyGuy/Neos-Headless-on-ARM-instructions I've also included precompiled binaries for Ampere A1 cpus (such as for the oracle cloud) and Raspberry Pis.

Enjoy!

PointerOffset commented 2 years ago

Tossed in my binaries for the M1 on Asahi as well for those that have a spare Apple Silicon Mac laying around.

FlameSoulis commented 1 year ago

Somehow, I recall someone mentioning that FreeImage was available as a package on Debian distros, but this was long ago and I don't have access to the support channels anymore.

I haven't dabbled with this for a while, so I'll first try at least on a Pi and see if those findings help with the VMs. As mentioned in my last post, FreeImage was the roadblock for me.

On Tue, Sep 13, 2022 at 2:13 PM Cyro @.***> wrote:

So while I've managed to get FreeImage to compile, I'm getting a message upon starting a world on the headless from mono:mono: symbol lookup error: /home/ubuntu/Neos Server/NeosVR/libFreeImage.so: undefined symbol: png_init_filter_functions_neon

Talking to geenz reveals that this is some sort of issue with an SIMD instructionset or something not being enabled (which I've looked up, and it appears that it should be enabled by default on aarch64)

The pngpriv.h file within Source/LibPNG details some of how to enable it, but I've tried setting CXXFLAGS += -DPNG_ARM_NEON_OPT=1 in Makefile.gnu in the main directory, along with trying to just set it on CFLAGS as well, but nothing I've tried so far makes even a lick of difference.

I was also told to look at the azure pipelines branch to get an idea of how compiling FreeImage is done on linux, but either I'm too inexperienced or the only thing related to azure pipelines is azure-pipelines.yml in the azure pipeline branch.

— Reply to this email directly, view it on GitHub https://github.com/Neos-Metaverse/NeosPublic/issues/2766#issuecomment-1245783267, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUE2ALPURIVOGP4JYMLTP3V6C75HANCNFSM5B3XKHVQ . You are receiving this because you commented.Message ID: @.***>

-- Danny Pearce Second Life: Flame Soulis/Swenholt Co-Owner of Kinzart Kreetures Founder of Forgotten Memories Creator of the Naverous Second Life Vendor Systems