Detect and warn when available system RAM is too low for LTO compilation to succeed in SCons

[LimaneGaya]

Tested versions

• Godot 4.3 dev [6a13fdc]

System information

kde neon 6.1, 6gb ram, apu

Issue description

I have been trying to compile Godot in the latest master when i compile the web export template it hangs with:

[ 76%] Linking Program bin/godot.web.template_release.wasm32.js ...
[ 78%] em++: warning: -pthread + ALLOW_MEMORY_GROWTH may run non-wasm code slowly, see https://github.com/WebAssembly/design/issues/1271 [-Wpthreads-mem-growth]
[ 96%] Building compilation database compile_commands.json
[100%] progress_finish(["progress_finish"], [])

it hangs like this for a really long time and i end up stopping it. I want to build the web release template by tripping some parts of the engine that i don't need and using encription. Without lto the template compiles but even with all the engine part stripped it still a little bigger than the default Godot 4.2 stable web template.

Here is the configuration i used:

scons platform=web target=template_release profile=custom.py

With custom.py containing the following:

production = "yes"
lto = "full"
optimize = "size"
deprecated = "no"
minizip = "no"
vulkan = "no"
module_bmp_enabled = "no"
module_camera_enabled = "no"
module_csg_enabled = "no"
module_enet_enabled = "no"
module_hdr_enabled = "no"
module_jsonrpc_enabled = "no"
module_mbedtls_enabled = "no"
module_minimp3_enabled = "no"
module_mobile_vr_enabled = "no"
module_multiplayer_enabled = "no"
module_openxr_enabled = "no"
module_theora_enabled = "no"
module_upnp_enabled = "no"
module_vhacd_enabled = "no"
module_webrtc_enabled = "no"
module_websocket_enabled = "no"
module_webxr_enabled = "no"
module_zip_enabled = "no"
module_text_server_fb_enabled = "yes"
module_text_server_adv_enabled = "no"
module_mono_enabled = "no"

emcc --version is 3.1.61

I'm new to compiling Godot and it take a long time to compile on my pc, i'm not sure if this is a bug or that it just take too long to compile.

Steps to reproduce

scons platform=web target=template_release profile=custom.py

Minimal reproduction project (MRP)

custom.zip

[Calinou]

6gb ram

You need at least 12 GB of system RAM to be able to use full LTO when compiling Godot. In practice, you typically need about 8 GB of available RAM when building, but since other apps on the system are also using RAM, it's often 12 GB in practice (16 GB if you have many apps open in the background).

Godot is a relatively large C++ application[^1], so it needs more memory to link than simpler C++ projects would.

[^1]: Not nearly as complex as Firefox, Chromium, LibreOffice, LLVM and so on still 🙂

[akien-mga]

it hangs like this for a really long time and i end up stopping it.

How long is "a really long time" in your case? LTO takes a really long time as a baseline, on our super powerful build server with 256 GB RAM and 64 cores it takes easily 20 min for web builds (sadly LLVM/Emscripten LTO only uses a single thread so it can't be parallelized despite 64 cores).

So on a less powerful machine it can easily take 40 min or even an hour. If you haven't waited this long, then you just need to wait longer and it might succeed.

[akien-mga]

I wonder if we can find a (not too hacky way) to display a message when the linking steps starts with LTO enabled, to let users know that this can take a while. WDYT @Repiteo ?

We could alternatively put this message at the start, printing from SConstruct directly, but this would be easy for users to overlook.

[LimaneGaya]

You need at least 12 GB of system RAM to be able to use full LTO

I have swap set to 8gb but i guess then it's not ideal, i have managed to reduce the size of the wasm from 50mb of the official 4.3beta2 to 37mb using lto=none, so it's still good and with gzip compression it gets to 7mb, however i wonder if there was a way to precompress to brotli since it gets it to 5mb.

How long is "a really long time" in your case?

I left it for an hour and half, but didn't have any feedback of what was going on from the terminal which let me think it was an issue. it was stuck on [100%] progress_finish(["progress_finish"], []) ( Reason i opened the issue ).

LTO takes a really long time as a baseline, on our super powerful build server with 256 GB RAM and 64 cores it takes easily 20 min for web builds (sadly LLVM/Emscripten LTO only uses a single thread so it can't be parallelized despite 64 cores).

That would take a really long then especially when testing what compiles to smallest size. How much does it help with size? i would guess even the official beta exports are using lto but have 50mb, i got it to 37mb without it (By disabling features but still), i would guess there is no major difference between them.

[akien-mga]

LTO is mostly used for its impact on performance, not size. If your main concern is size I'm not sure it makes a big difference (maybe up to 5%).

[Calinou]

I have swap set to 8gb but i guess then it's not ideal, i have managed to reduce the size of the wasm from 50mb of the official 4.3beta2 to 37mb using lto=none, so it's still good and with gzip compression it gets to 7mb, however i wonder if there was a way to precompress to brotli since it gets it to 5mb.

You can use Brotli precompression, but the web server needs to be configured to serve the precompressed .wasm file with the .wasm.br file extension. This can't be done if using itch.io or GitHub Pages as a host for your project.

[Calinou]

Detecting when available system RAM is low (or even the total system RAM amount) doesn't seem to be possible without a library that isn't available in Python's standard library, psutil: https://stackoverflow.com/questions/276052/how-to-get-current-cpu-and-ram-usage-in-python

We could technically vendor this library in Godot's source tree so users don't have to install it through pip, but this feels overkill.

Something like this in the top-level SConstruct:

    if env["lto"] == "full":
        import psutil

        memory_available_mib = psutil.virtual_memory().available / 1048576
        if memory_available_mib < 8192:
            print_warning(f"Detected less than 8 GiB of RAM being available when starting compilation (currently available: {round(memory_available_mib)} MiB). Linking with `lto=full` may fail; try `lto=thin` or `lto=none` instead.")

I also can't get the SCons options to show up in the env parameter that is passed to progress_finish in https://github.com/godotengine/godot/blob/b97110cd307e4d78e20bfafe5de6c082194b2cd6/methods.py#L982, so that we could display a notice conditionally when LTO is enabled.