FreeBSD support is missing

[graemeg]

Environment

• LWJGL version: 3.2
• LWJGL build #: unknown
• Java version: openjdk 8
• Platform: FreeBSD 11.1
• Module: core

Description

The Minecraft 1.12.x game worked just fine under FreeBSD with lwjgl 2.9.x but then when Minecraft 1.13.x came out, they moved up to lwjgl 3.x. Currently the lwjgl 3.x doesn't have support for FreeBSD.

Seeing as v2.9.x did, is there any changes somebody here could try and add v3.x support for FreeBSD, so we can get the wonderful Minecraft game working again. :-)

Somebody in the FreeBSD web forums have tried to get lwjgl 3.2 to build under FreeBSD 11.1, and here is how far they can. Hopefully this is a good start for anybody with more knowledge, to add the missing piece, or understand the compilation errors. I don't.

https://forums.freebsd.org/threads/minecraft-11-2-failed-to-locate-library-liblwjgl-so.66923/#post-402189

Here is a copy of that forum message:

With some help on #freebsd freenode, I am attempting to compile lwjgl 3.2 on FreeBSD 11.1-STABLE. I have modified lwjgl's build.xml file to compile using gcc8.

https://github.com/LWJGL/lwjgl3/blob/2359eb7b5a9e7e629beae95d3cdab8e997c59039/config/linux/build.xml

And here is my modified build.xml, to work with gcc8:

https://paste.pound-python.org/show/Akuv2tbz32OTgSyqZKfJ/

Using the build instruction found here: https://www.lwjgl.org/guide#build-instructions

I am getting the following output: https://paste.pound-python.org/show/8GO6dhwjNrQFu1ddtx2a/

[Spasi]

You can find an attempt to port LWJGL (at least the parts Minecraft needs) to FreeBSD here.

Official support won't be possible without a CI service (like Travis, AppVeyor, etc.) that supports FreeBSD. Afaik, such a service does not exist and cross-compilation from Linux is painful. If someone would like to host a solution (Jenkins or whatever) for us, please let me know.

[nkfilis]

@Spasi we could be willing to provide that, what would you need there?

[Spasi]

Hey @nkfilis,

A FreeBSD environment with the tools (GCC, git, CMake, etc.) and development headers necessary to build the projects listed here. You can check the .travis.yml file in each repository to see what's involved. Currently the build workflow is:

• I push latest version to an LWJGL-CI repository.
• This triggers a new build at Travis CI (Linux & macOS) and AppVeyor (Windows).
• When the build is done, the new artifacts are uploaded to LWJGL's S3 bucket.

Ideally, the FreeBSD solution would work as simply as that.

[graemeg]

Just to let you know, I followed those instructions and managed to compile LWJGL3 and what was needed for Minecraft 1.13.2. It worked perfectly fine on my FreeBSD 11.2 64-bit system.

[graemeg]

If @nkfilis can't manage the CI setup for FreeBSD, I'll happily give it a go too. My FreeBSD server already runs 24/7 for other open source projects.

[lwhsu]

I think this page could help a little bit about FreeBSD CI: https://wiki.freebsd.org/HostedCI

[kmosiejczuk]

Official support won't be possible without a CI service (like Travis, AppVeyor, etc.) that supports FreeBSD. Afaik, such a service does not exist and cross-compilation from Linux is painful. If someone would like to host a solution (Jenkins or whatever) for us, please let me know.

FYI, Sourcehut supports FreeBSD and OpenBSD CI: https://man.sr.ht/builds.sr.ht/compatibility.md

I haven't used it, but I know other OpenBSD developers who said it works quite well.

[stormchaser3000]

would anyone be willing to layout what it would take to configure a CI system to support FreeBSD? I would be willing to learn and help with it, but my knowledge at the moment is rather low.

[lwhsu]

Please check https://wiki.freebsd.org/HostedCI , there is some examples, and TravisCI also has experimental FreeBSD support now.

[Spasi]

LWJGL-CI has migrated to GitHub Actions. Anyone still interested in FreeBSD support, you may want to try https://github.com/marketplace/actions/freebsd-vm. Let me know if it works out please.

[ivan-volnov]

I'm looking forward to seeing freebsd support added

[ivan-volnov]

@Spasi Added a PR for FreeBSD Github Actions to LWJGL-CI

[yurivict]

CI aside, does Gephi's master branch build/work under FreeBSD?

[grahamperrin]

Started attempt at making a lwjgl3 port : freebsd (2021-05-14)

Working port of lwjgl3 to FreeBSD (2021-07-09)

[grahamperrin]

⚙ D31119 games/lwjgl3: Lightweight Java Game Library 3

[ivan-volnov]

@grahamperrin great work! If everything is ok, post a pull request to the upstream

[grahamperrin]

Not my work, I'm just linking

[Irgendwer01]

@ivan-volnov that's the github of them who worked on LWJGL3 support for FreeBSD https://github.com/CRKatri/lwjgl3

[qolarnix]

Hello, what is the status of FreeBSD support? Has there been any progress?

I have tried to compile using forks that claim they work but no luck. I assume either my environment is not setup correctly or the previous methods no longer work.

[graemeg]

I'm on FreeBSD 13.1 and the forks work for Minecraft up to MC v1.19.2 without issue. I'm using the [https://github.com/patrickSpaceSurfer/lwjgl3-port] fork.

The Minecraft 1.19.3 release uses a new LWJGL API call org.lwjgl.glfw.GLFWImage.malloc(int intArg, org.lwjgl.system.MemoryStack memStack) that isn't in the forks. I tried to backport that API call, but the code that auto-generates the JNI java code, doesn't seem to see my change for some reason. To be honest, I haven't tried very hard to get it working, as there isn't a big change in 1.19.2 and 1.19.3 (regarding game features). I do want to look at it again though, before the 1.20 Minecraft release comes out. :-)

On a side note: I am working on implementing full FreeBSD support in LWJGL though (not using the "linux" workaround that the forks do) - but my motivation is low, seeing as the LWJGL developers aren't really interested in supporting FreeBSD at all. Multiple people (including myself) offered to supply a CI build environment etc, but still nothing. So us FreeBSD folks will have to keep at it the hard way. :-(

[Spasi]

my motivation is low, seeing as the LWJGL developers aren't really interested in supporting FreeBSD at all

This is not accurate. We obviously want LWJGL to be usable on more platforms, but every platform/architecture added means increased maintenance burden. Every platform has its quirks (e.g. the libc nightmare on Linux) and LWJGL has a lot of dependencies with all kinds of weirdness, build systems, etc. It has to be worth the effort. Please, don't forget that LWJGL is maintained by a single person.

But, hey, good news! LWJGL-CI accepts contributions! We already have https://github.com/LWJGL-CI/lwjgl3/pull/3 and I'm sure we can get it to work. The FreeBSD-VM-on-macOS-runner solution seems to be viable. Doing the same for some critical dependencies (e.g. libffi, GLFW, OpenAL-Soft) would be a great step towards adding official support for FreeBSD.

[qolarnix]

Wow I did not expect to see this kind of response so quickly. Glad to see that there are still people interested in getting this working. I know that this will benefit a lot of people. @graemeg thank you for recommending the patrickSpaceSurfer port, I will give that a try.

[qolarnix]

Not sure if this is the right place to ask (if not please redirect me) but I seem to have the same issue with every fork that I've tried so far:

    [mkdir] Created dir: /usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/bin/linux/x64

compile-native-platform:
    [mkdir] Created dir: /usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/bin/linux/x64/libdivide
    [mkdir] Created dir: /usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/bin/linux/x64/opengles
    [mkdir] Created dir: /usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/bin/linux/x64/stb
    [mkdir] Created dir: /usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/bin/linux/x64/opengl
    [mkdir] Created dir: /usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/bin/linux/x64/llvm
    [mkdir] Created dir: /usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/bin/linux/x64/core
       [cc] Linking lwjgl_libdivide
       [cc] Linking lwjgl_llvm
 [Compiler] /usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/modules/lwjgl/core/src/generated/c/org_lwjgl_system_dyncall_DynCall.c:91:5: fatal error: use of undeclared identifier 'DCstruct'; did you mean 'struct'?
 [Compiler]     DCstruct *s = (DCstruct *)(intptr_t)sAddress;
 [Compiler]     ^~~~~~~~
 [Compiler]     struct
 [Compiler] 1 error generated.
       [cc] Linking lwjgl_opengles
       [cc] Linking lwjgl_opengl
       [cc] Linking lwjgl_stb

BUILD FAILED
/usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/build.xml:406: The following error occurred while executing this line:
/usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/config/linux/build.xml:157: The following error occurred while executing this line:
/usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/config/linux/build.xml:92: The following error occurred while executing this line:
/usr/home/polarnix/Git/projects/lwjgl3-port/lwjgl3/config/linux/build.xml:39: apply returned: 1

I'm not quite sure what I'm missing here. if I could get some guidance it would be appreciated.

[Spasi]

Hey @qolarnix,

Looks like you're on an old LWJGL version, or haven't cleaned the project since upgrading. The dyncall bindings do not exist anymore, they have been replaced with libffi.

[graemeg]

Yes, the "minecraft" forks still use LWJGL 3.2.2 (or 3.2.3). To fix the DCstruct issue of FreeBSD, downgrade your DynCall package to v1.3 (I'm assuming it's currently on v1.4 - latest). I had to do this as well last night, when I played around with LWJGL+FreeBSD branch.

If you are using Ports, I simply made a backup of the ports dyncall directory, then modified it to point to the v1.3 release and SHA1 value (values and url can be found here: https://dyncall.org/download). Then make and install the port as usual. If you are using binary pkg packages, you might have the old package in your /var/cache/pkg/ directory - then simply pkg install it.

[graemeg]

Hi @Spasi

Thank you for your reply, and it's very good news to know that a workable FreeBSD CI is available. I'll keep working on my FreeBSD support and let you know when it's done/working.

Please, don't forget that LWJGL is maintained by a single person.

My apologies, I didn't know that. I guess the "games written in Java" are still small, compare do other languages. :-(

[graemeg]

Glad to see that there are still people interested in getting this working

For sure. :+1: I have done many comparisons/tests between Linux and FreeBSD on my same hardware. I get way more consistent, and higher FPS under FreeBSD and less lag spikes. So FreeBSD still stays my favourite operating system, to run my favourite game. :-D

As for the various minecraft lwjgl forks. They all pretty much do the same thing. Someone (can't remember who) just realised that they can tweak the minecraft runtime script (included with those forks) so it ignores the exact LWJGL version that MC needs. Minecraft 1.17 -> 1.19.2 don't seem to use different API calls, even though Minecraft changed the LWJGL versions they use. So the original LWJGL 3.2.2 still works. But as I mentioned, we need to update it for 1.19.3 to work - I'm on it!

[graemeg]

@qolarnix If you don't come right compiling one of the lwjgl-freebsd forks, I've compiled and packaged LWJGL 3.2.3 for FreeBSD 13.1 (amd64). You can download it here: http://geldenhuys.co.uk/~graemeg/minecraft-freebsd/ (yes, I don't have a SSL certificate) Simply unpack, edit the ROOT path inside the minecraft-runtime file, and use that file as the "java executable" in your MC launcher (eg: MultiMC). I have OpenAL & GLFW installed from FreeBSD's binary package system.

[qolarnix]

@graemeg Thank you - this seems to remove the errors that I get regarding lwjgl. I am getting one last error when attempting to launch the game:

java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.base/java.lang.reflect.Method.invoke(Unknown Source)
    at org.multimc.impl.OneSixLauncher.invokeMain(OneSixLauncher.java:104)
    at org.multimc.impl.OneSixLauncher.launchWithMainClass(OneSixLauncher.java:175)
    at org.multimc.impl.OneSixLauncher.launch(OneSixLauncher.java:185)
    at org.multimc.EntryPoint.listen(EntryPoint.java:127)
    at org.multimc.EntryPoint.main(EntryPoint.java:57)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.mojang.blaze3d.systems.RenderSystem
    at z.a(SourceFile:66)
    at ejf.a(SourceFile:2408)
    at ejf.a(SourceFile:2403)
    at net.minecraft.client.main.Main.a(SourceFile:214)
    at net.minecraft.client.main.Main.main(SourceFile:51)
    ... 9 more

I am working on a solution but no luck so far. It seems that com.mojang.blaze3d.systems.RenderSystem is giving me issues no matter what launcher I use (portablemc, multimc).

I am using openjdk17, ive also tested with openjdk18.

[qolarnix]

Ok please disregard the above I had to change: ROOT=/home/polarnix/Git/projects/lwjgl-3.2.3/lwjgl I was missing "/home/polarnix/" and that solved the issue.

I got 1.19.2 to launch! Thanks!

[graemeg]

I've created a new release for 3.2.3 with the backported GLFW method - required to get MC 1.19.3 to work. :partying_face: You can download it here: http://geldenhuys.co.uk/~graemeg/minecraft-freebsd/lwjgl-3.2.3.1-freebsd.tar.7z

It works for the 1.19.4-pre1 release as well.

[qolarnix]

@graemeg You are a legend. I haven't had the chance to test out this build but I will keep you posted if I run into any issues. Please let me know if there is anything I can do to assist in getting this library fully ported, I am happy to test any builds that you send my way. Appreciate it a lot!

[ivan-volnov]

@graemeg Can you update FreeBSD port with your patch?

[NotAlexNoyle]

Bump. What is the status on a FreeBSD lwjgl3 port?

[SamuelGogarty]

wow, so this issue is still present? I remember posting on the forums about this in 2018

[Alex6357]

So is FreeBSD support in progress?

[graemeg]

I'm working on it in my personal capacity, so I can play Minecraft 2.20.2 onwards. This time I'm not backporting the missing functions to a older LWJGL [what I did for MC 1.19.4 to work]. I'm trying a native port with the latest code. I'm going to publish a personal branch this week, and hope the LWJGL devs (or general community) could possibly give me some guidance with some of the C/C++ compiler errors I'm getting during the compile-native-platform stage of building LWJGL. I'm not a C/C++ dev at all, so the compiler errors are somewhat confusing. :face_with_spiral_eyes:

[er2off]

I built lwjgl3 version 3.3.3 (latest at the moment)

I was lazy to learn ant build system so custom cmake file is used here just to build newer glfw (release one doesn't work)

Most modules were built which is more than enough to run latest version of minecraft

lwjgl3.tar.gz

[Spasi]

I've been testing a few different options for VM-based FreeBSD builds on Github Actions. It looks like cross-platform-actions/action is a solution with reasonable performance (xhyve on macOS runners, hardware accelerated virtualization).

I tried it on a light dependency (libffi) and a heavy one (Assimp), with acceptable results. See the freebsd-cross jobs for details. If you'd like to see this going faster, you're welcome to use these jobs as reference and contribute PRs for the other LWJGL dependencies.

One problem we have on Linux is having to build on old distributions for wider GLIBC compatibility. Is there a similar problem on FreeBSD? Do we have to use an older OS and/or compiler for compatibility? The examples above currently run on FreeBSD 13.2 (the latest?) with GCC 11. I also read something about GCC being deprecated on FreeBSD and Clang being the preferred choice.

[graemeg]

I don't know too much about C/C++ etc, but looking at FreeBSD's ports tree, where you can manually build and install software (the alternative to pre-built binary packages), I can see both libffi 3.4.4 and assimp 5.2.5 are included in the ports tree, using the latest released versions of those libraries, and doesn't require any FreeBSD patches to compile.

Compiler wise,

libffi uses: USES= compiler:c11 cpe libtool pathfix assimp uses: USES= cmake:testing compiler:c++17-lang

So it doesn't seem like FreeBSD requires all packages to be compiled with a single compiler. A mix of compilers are acceptable. Not sure if that helps you much.

[ivan-volnov]

FreeBSD 13+ uses clang as a main compiler. It's much preferable. If we can build it with clang it would be great.

[er2off]

I've been testing a few different options for VM-based FreeBSD builds on Github Actions. It looks like cross-platform-actions/action is a solution with reasonable performance (xhyve on macOS runners, hardware accelerated virtualization).

It is possible of course, I tested it before, but it prepares to start for a very long time.

I tried it on a light dependency (libffi) and a heavy one (Assimp), with acceptable results. See the freebsd-cross jobs for details. If you'd like to see this going faster, you're welcome to use these jobs as reference and contribute PRs for the other LWJGL dependencies.

This is good but clang instead of gcc will be better because it is included by default since FreeBSD 10.

One problem we have on Linux is having to build on old distributions for wider GLIBC compatibility. Is there a similar problem on FreeBSD? Do we have to use an older OS and/or compiler for compatibility? The examples above currently run on FreeBSD 13.2 (the latest?) with GCC 11. I also read something about GCC being deprecated on FreeBSD and Clang being the preferred choice.

I know problem with kernel modules but not with programs. Most of them should correctly run on FreeBSD and it doesn't use glibc. FreeBSD have less supported releases, which are currently 12.4 and 13.2, today 14.0 release builds was started. Also as I previously mentioned, Clang is default compiler.

[Spasi]

Hey @er2off,

but it prepares to start for a very long time.

This happened in my tests too. Turns out that the action launches the VM with access to the entire runner memory by default, which causes instability. I haven't seen anything weird after limiting it to something reasonable (4G atm).

FreeBSD have less supported releases, which are currently 12.4 and 13.2, today 14.0 release builds was started. Also as I previously mentioned, Clang is default compiler.

I'll proceed with the assumption that 13.2 and whatever Clang version (14?) is pre-installed there is OK. We'll see if this needs adjusting after a complete LWJGL build is available and users test it.

Progress update: Switched to Clang compilation and we now also have jemalloc and GLFW builds. See https://www.lwjgl.org/browse/nightly/freebsd/x64.

[er2off]

Turns out that the action launches the VM with access to the entire runner memory by default, which causes instability. I haven't seen anything weird after limiting it to something reasonable (4G atm).

Will be nice if GitHub Actions will add official FreeBSD image. As for now, alternative is to use other CI (such as Cirrus CI) or even cross compiling using clang.

Also it's not worth to implement x86 build because 32bit support will be dropped in FreeBSD 15.0 (just like macOS).

I'll proceed with the assumption that 13.2 and whatever Clang version (14?) is pre-installed there is OK. We'll see if this needs adjusting after a complete LWJGL build is available and users test it.

Pre-installed clang compiler is enough. Base FreeBSD installation can build its source code without installing anything.

Progress update: Switched to Clang compilation and we now also have jemalloc and GLFW builds. See https://www.lwjgl.org/browse/nightly/freebsd/x64.

It is very nice, upstream support is closer. Also jemalloc is system allocator for a very long time. I don't know if jemalloc library is required.

Edit: I updated my port to support downloadable libraries

[er2off]

I tested jemalloc library and it crashes jvm when running tests so I currently disabled this library.

Also shaderc library in ports have different name (it's libshaderc_shared.so) so in my patches I have to add library name mapper in modules/lwjgl/core/src/main/java/org/lwjgl/system/Platform.java (this is explanation for someone who will look inside my patches in archive above)

BTW tests.txt without libjemalloc used, I don't like these warnings.

[Spasi]

@er2off

I tested jemalloc library and it crashes jvm when running tests so I currently disabled this library.

Could you post more details please? I don't have a FreeBSD system available so can't test any of the binaries. Even if jemalloc is the default allocator on FreeBSD, the binary provided by LWJGL should work and can be useful (specific version, can be replaced with custom build that has different features enabled).

Also shaderc library in ports have different name (it's libshaderc_shared.so)

Fixed.

BTW tests.txt without libjemalloc used, I don't like these warnings.

LWJGL tries to load the jemalloc module by default, before falling back to the system allocator. You can use -Dorg.lwjgl.system.allocator=system to skip jemalloc and the warning will go away.

Progress update: all non-core dependencies are now available for FreeBSD, added:

• bgfx
• FreeType
• HarfBuzz
• hwloc
• KTX
• OpenAL Soft
• OpenXR Loader
• Opus
• shaderc
• SPIRV-Cross

[er2off]

@Spasi

Could you post more details please? I don't have a FreeBSD system available so can't test any of the binaries. Even if jemalloc is the default allocator on FreeBSD, the binary provided by LWJGL should work and can be useful (specific version, can be replaced with custom build that has different features enabled).

Hm, I recompiled with new dependencies and now I have in tests error about pthread:

    [Tests] [LWJGL] Loading library: jemalloc
    [Tests]     Module: org.lwjgl.jemalloc
    [Tests] <jemalloc>: Error in dlsym(RTLD_NEXT, "pthread_create")

Build is somehow marked as successful but there is no test statistics (no, jvm also crashes with jemalloc, it wasn't shown for me).

Also I use in ports makefile -Dkotlinc and kotlin as build dependency. Should I change this to use downloadable kotlinc instead?

[Spasi]

@er2off

error about pthread

Thanks, that helped. Could you try the latest build please?

I use in ports makefile -Dkotlinc and kotlin as build dependency

Not sure how FreeBSD ports work, but the Ant build downloads kotlinc automatically. You shouldn't have to download it with another mechanism. Kotlin is used for code generation only, during the build process, it's not an LWJGL dependency.

[er2off]

@Spasi

Could you try the latest build please?

Tests are now failed. I don't sure why, jvm crashes when try to start any game on lwjgl.

FreeBSD have patched jemalloc, idk why it was patched.

You shouldn't have to download it with another mechanism.

So now it downloads as usual.

[Spasi]

I installed FreeBSD on an old machine, resolved the jemalloc issue and made all necessary changes to the LWJGL core. All tests pass and demos work great.

Thanks everyone, 3.3.4-snapshot+2 will include FreeBSD artifacts!