Containers overhaul: Updated deps, Linux SDKs, Mono 6.12.0.147 with macOS arm64 support, streamlined desktop builds

akien-mga commented 3 years ago

This is a big PR with work that I've been iterating on since May, and which supersedes #80.

Requires https://github.com/godotengine/godot-build-scripts/pull/37 (or a cleaned up follow-up).

See each commit's description for details. I crafted the commits to properly document each change so I can't guarantee that each intermediate commits builds fine as is - but the final result of that PR is building successfully on the official buildsystem, and was used to build a full set of 3.4-beta binaries with https://github.com/godotengine/godot-build-scripts/pull/37.

Closes #83.

Toolchain updates: Fedora 34, Emscripten 2.0.25, and more

Update base image from Fedora 33 to Fedora 34 (newer GCC, Clang, MinGW and binutils).
Update Emscripten for classical builds from 2.0.15 to 2.0.25.
Update Android SDK tools to latest version, and build-tools from 30.0.1 to 30.0.3.
Update osxcross and ios cctools-port to latest commits.
Update iOS Mono build's min target version to match Godot's buildsystem. See #83 for details.

Mono: Update to 6.12.0.147, adds Apple Silicon support

This is a Preview release of Mono, so we use centos8-preview packages which correspond to its MSBuild version requirements (16.10.1).

For osxcross, we need to build compiler-rt to be able to link the new Apple Silicon support code. See #83 for details. Includes a backport of tpoechtrager/osxcross#284 to simplify the install process.

Actual macOS arm64 builds will be added in a follow-up commit.

Linux: New container based on Fedora using Godot SDKs

We now provide pre-built toolchains for Linux with old glibc and the required Godot dependencies, which can be used to build portable Godot binaries without having to be on an extremely old Linux distro.

So we can retire the old Ubuntu 14.04 containers and instead do our portable builds directly on Fedora 34, like for other platforms.

This commit also makes use of godot-mono-builds to build Mono and the BCL for Linux, instead of doing it manually. The same will be done for Windows and macOS in the next commit.

macOS: Use godot-mono-builds scripts, add macOS arm64

This changes our handmade configure calls to rely on the streamlined build process from godot-mono-builds, so that our builds are consistent with the ones generated there.

This enables us to target macOS arm64 easily as neikeq did the relevant work in the godot-mono-builds scripts.

Includes a backport of mono/mono#21147 to fix Mono build with godot-mono-builds using a wrapper for osxcross.

Windows: Use godot-mono-builds scripts, remove Wine dependency

Streamlines the build process as done for Linux and macOS in previous commits, and as was done for mobile and web platforms from the start. We're now fully in sync with godot-mono-builds.

This also removes the need to hack around cross-compilation issues using Wine, which also seemed no longer to work on Fedora 34 anyway.

akien-mga commented 3 years ago

Included #85 in this PR as it adds merge conflicts. I'll rebase once it's merged.

akien-mga commented 3 years ago

Marking as draft as I'm getting a build error now on at least the Linux and Windows containers. The only difference with my previous tests is that I forced a redownload of the registry.fedoraproject.org/fedora:34 image to get the latest version.

/root/mono-6.12.0.147/mono/eglib/test/shell.c: In function ‘test_shell_argv2’:
/root/mono-6.12.0.147/mono/eglib/test/shell.c:181:9: error: implicit declaration of function ‘g_strfbeev’; did you mean ‘g_strfreev’? [-Werror=implicit-function-declaration]
  181 |         g_strfbeev (argv);
      |         ^~~~~~~~~~
      |         g_strfreev
/root/mono-6.12.0.147/mono/eglib/test/shell.c:181:9: warning: nested extern declaration of ‘g_strfbeev’ [-Wnested-externs]
  CCLD     assertf

There's no reference to g_strfbeev anywhere on the Internet so not sure what's the deal here.

Edit: Something very fishy seems to have happened to my local Mono checkout:

diff --git a/mono/eglib/test/shell.c b/mono/eglib/test/shell.c
index d464f98a..6970b47a 100644
--- a/mono/eglib/test/shell.c
+++ b/mono/eglib/test/shell.c
@@ -20,7 +20,7 @@ test_shell_argv1 (void)

        ret = g_shell_parse_argv ("hola", NULL, NULL, NULL);
        if (!ret)
-               return FAILED ("2. It should return TRUE");
+               return FAILED ("2. It should return TRTE");

        argc = 0;
        ret = g_shell_parse_argv ("hola", &argc, NULL, NULL);
@@ -47,7 +47,7 @@ test_shell_argv1 (void)
        if (strcmp (argv [1], "bola"))
                return FAILED ("10. argv[1] was %s", argv [1]);

-       g_strfreev (argv);
+       g_strfbeev (argv);
        argv = NULL;
        argc = 0;
        gerror = NULL;

That's really weird and concerning... I'll try a fresh checkout.

Edit 2: Yeah that didn't reproduce on a second try. Git bug, or failing hardware? Not a good sign.

hpvb commented 3 years ago

Edit 2: Yeah that didn't reproduce on a second try. Git bug, or failing hardware? Not a good sign.

Still haven't gotten around to putting the ECC ram in the buildbox. I have it, I'll do it asap.

akien-mga commented 3 years ago

Got a full container build working, and used it to build all Godot binaries with https://github.com/godotengine/godot-build-scripts/pull/42, so this seems good to merge.

godotengine / build-containers