search

godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
91.13k stars 21.19k forks source link

Compiling Godot web export template with LTO fails with Emscripten 3.1.42+ #80010

Closed radlotus closed 1 year ago

radlotus commented 1 year ago

Godot version

3.5.2

System information

AMD EPYC 7B13; 32 GB RAM Ubuntu 22.04.2 LTS; Python 3.11.1; Emscripten 3.1.44; clang 17; Scons 4.0.1

Issue description

Compiling with LTO gives this error:

[Initial build] Compiling ==> core/bind/core_bind.cpp
[Initial build] Creating 'bin/godot.javascript.opt.engine.js'
[Initial build] Install file: "misc/dist/html/offline-export.html" as "bin/.javascript_zip/godot.offline.html"
[Initial build] Install file: "misc/dist/html/service-worker.js" as "bin/.javascript_zip/godot.service.worker.js"
[Initial build] progress_finish(["progress_finish"], [])
[Initial build] Linking Static Library ==> core/libcore.javascript.opt.a
Ranlib Library         ==> core/libcore.javascript.opt.a
[Initial build] Linking Program        ==> bin/godot.javascript.opt.js
cache:INFO: generating system asset: symbol_lists/df45337bb9df6876919ba6b0bb4825c9a26cbd2f.json... (this will be cached in "/workspace/web/emsdk/upstream/emscripten/cache/symbol_lists/df45337bb9df6876919ba6b0bb4825c9a26cbd2f.json" for subsequent builds)
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libGL-webgl2-ofb.a... (this will be cached in "/workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libGL-webgl2-ofb.a" for subsequent builds)
system_libs:INFO: compiled 4 inputs
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libal.a... (this will be cached in "/workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libal.a" for subsequent builds)
system_libs:INFO: compiled 1 inputs
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libhtml5.a... (this will be cached in "/workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libhtml5.a" for subsequent builds)
system_libs:INFO: compiled 3 inputs
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libstubs.a... (this will be cached in "/workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libstubs.a" for subsequent builds)
system_libs:INFO: compiled 2 inputs
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libc.a... (this will be cached in "/workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libc.a" for subsequent builds)
system_libs:INFO: compiled 1023 inputs
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libdlmalloc.a... (this will be cached in "/workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libdlmalloc.a" for subsequent builds)
system_libs:INFO: compiled 1 inputs
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libcompiler_rt.a... (this will be cached in "/workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libcompiler_rt.a" for subsequent builds)
system_libs:INFO: compiled 175 inputs
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libc++-noexcept.a... (this will be cached in "/workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libc++-noexcept.a" for subsequent builds)
system_libs:INFO: compiled 49 inputs
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libc++abi-noexcept.a... (this will be cached in "/workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libc++abi-noexcept.a" for subsequent builds)
system_libs:INFO: compiled 16 inputs
cache:INFO:  - ok
cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/lto/libsockets.a... (this will be cached in "/workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libsockets.a" for subsequent builds)
system_libs:INFO: compiled 21 inputs
cache:INFO:  - ok
wasm-ld: error: /workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libc.a(scalbnf.o): attempt to add bitcode file after LTO.
em++: error: '/workspace/web/emsdk/upstream/bin/wasm-ld -o bin/godot.javascript.opt.wasm platform/javascript/audio_driver_javascript.javascript.opt.o platform/javascript/godot_webgl2.javascript.opt.o platform/javascript/http_client_javascript.javascript.opt.o platform/javascript/javascript_singleton.javascript.opt.o platform/javascript/javascript_main.javascript.opt.o platform/javascript/os_javascript.javascript.opt.o platform/javascript/api/javascript_tools_editor_plugin.javascript.opt.o platform/javascript/javascript_runtime.javascript.opt.o main/libmain.javascript.opt.a modules/libmodules.javascript.opt.a platform/libplatform.javascript.opt.a drivers/libdrivers.javascript.opt.a scene/libscene.javascript.opt.a servers/libservers.javascript.opt.a core/libcore.javascript.opt.a modules/freetype/libfreetype_builtin.javascript.opt.a -L/workspace/web/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto -lGL-webgl2-ofb -lal -lhtml5 -lstubs -lc -ldlmalloc -lcompiler_rt -lc++-noexcept -lc++abi-noexcept -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr /tmp/tmpk0a0yca9libemscripten_js_symbols.so --strip-debug --export-if-defined=main --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_lib_deps --export-if-defined=__stop_em_lib_deps --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-if-defined=__main_argc_argv --export-if-defined=fflush --export=stackSave --export=stackRestore --export=stackAlloc --export=__errno_location --export=__get_temp_ret --export=__set_temp_ret --export=__funcs_on_exit --export=__wasm_call_ctors --export-table -z stack-size=65536 --initial-memory=33554432 --no-entry --max-memory=2147483648 --global-base=1024' failed (returned 1)
scons: *** [bin/godot.javascript.opt.js] Error 1
scons: building terminated because of errors

Steps to reproduce

  1. Install the requirements listed on https://docs.godotengine.org/en/3.6/development/compiling/compiling_for_web.html.
  2. Run scons platform=javascript tools=no target=release use_lto=yes

Does anyone else have this issue?

Is it something related to a newer version of Emscripten? Python? Scons?

I did not have this issue before. I am using a fresh build setup.

radlotus commented 1 year ago

whoops, closed accidentaly.

akien-mga commented 1 year ago

Official builds are made with Emscripten 3.1.18. Newer versions often cause breaking regressions, so this might be the case too. Could you test with 3.1.18 and see if that solves it?

radlotus commented 1 year ago

@akien-mga, yes, using 3.1.18 solves it. I've checked and the latest version that still works is 3.1.36.

radlotus commented 1 year ago

The issue is also present in Godot 4. Last working Emscripten version is also 3.1.36

Faless commented 1 year ago

This seems to be an upstream bug, see: https://github.com/emscripten-core/emscripten/issues/19774 and https://github.com/emscripten-core/emscripten/issues/19781 .

akien-mga commented 1 year ago

For the record, I can confirm the issue is still reproducible with latest Emscripten 3.1.45.

Tested build with scons p=web target=template_release production=yes verbose=yes. Fails with:

wasm-ld: error: /home/akien/Projects/godot/emscripten/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/libc-mt-debug.a(scalbnf.o): attempt to add bitcode file after LTO.
em++: error: '/home/akien/Projects/godot/emscripten/emsdk/upstream/bin/wasm-ld -o bin/godot.web.template_release.wasm32.wasm platform/web/audio_driver_web.web.template_release.wasm32.o platform/web/display_server_web.web.template_release.wasm32.o platform/web/http_client_web.web.template_release.wasm32.o platform/web/javascript_bridge_singleton.web.template_release.wasm32.o platform/web/web_main.web.template_release.wasm32.o platform/web/os_web.web.template_release.wasm32.o platform/web/api/web_tools_editor_plugin.web.template_release.wasm32.o platform/web/web_runtime.web.template_release.wasm32.o main/libmain.web.template_release.wasm32.a modules/libmodules.web.template_release.wasm32.a modules/libmodule_ktx.web.template_release.wasm32.a modules/libmodule_msdfgen.web.template_release.wasm32.a modules/libmodule_theora.web.template_release.wasm32.a modules/libmodule_vorbis.web.template_release.wasm32.a modules/libmodule_basis_universal.web.template_release.wasm32.a modules/libmodule_bmp.web.template_release.wasm32.a modules/libmodule_csg.web.template_release.wasm32.a modules/libmodule_dds.web.template_release.wasm32.a modules/libmodule_enet.web.template_release.wasm32.a modules/libmodule_freetype.web.template_release.wasm32.a modules/libmodule_gdscript.web.template_release.wasm32.a modules/libmodule_glslang.web.template_release.wasm32.a modules/libmodule_gltf.web.template_release.wasm32.a modules/libmodule_gridmap.web.template_release.wasm32.a modules/libmodule_hdr.web.template_release.wasm32.a modules/libmodule_jpg.web.template_release.wasm32.a modules/libmodule_jsonrpc.web.template_release.wasm32.a modules/libmodule_lightmapper_rd.web.template_release.wasm32.a modules/libmodule_mbedtls.web.template_release.wasm32.a modules/libmodule_meshoptimizer.web.template_release.wasm32.a modules/libmodule_minimp3.web.template_release.wasm32.a modules/libmodule_mobile_vr.web.template_release.wasm32.a modules/libmodule_multiplayer.web.template_release.wasm32.a modules/libmodule_navigation.web.template_release.wasm32.a modules/libmodule_noise.web.template_release.wasm32.a modules/libmodule_ogg.web.template_release.wasm32.a modules/libmodule_raycast.web.template_release.wasm32.a modules/libmodule_regex.web.template_release.wasm32.a modules/libmodule_squish.web.template_release.wasm32.a modules/libmodule_svg.web.template_release.wasm32.a modules/libmodule_text_server_adv.web.template_release.wasm32.a modules/libmodule_tga.web.template_release.wasm32.a modules/libmodule_upnp.web.template_release.wasm32.a modules/libmodule_vhacd.web.template_release.wasm32.a modules/libmodule_webp.web.template_release.wasm32.a modules/libmodule_webrtc.web.template_release.wasm32.a modules/libmodule_websocket.web.template_release.wasm32.a modules/libmodule_webxr.web.template_release.wasm32.a modules/libmodule_zip.web.template_release.wasm32.a platform/libplatform.web.template_release.wasm32.a drivers/libdrivers.web.template_release.wasm32.a scene/libscene.web.template_release.wasm32.a servers/libservers.web.template_release.wasm32.a core/libcore.web.template_release.wasm32.a modules/text_server_adv/libharfbuzz_builtin.web.template_release.wasm32.a modules/text_server_adv/libgraphite_builtin.web.template_release.wasm32.a modules/text_server_adv/libicu_builtin.web.template_release.wasm32.a modules/freetype/libfreetype_builtin.web.template_release.wasm32.a modules/msdfgen/libmsdfgen_builtin.web.template_release.wasm32.a -L/home/akien/Projects/godot/emscripten/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto /home/akien/Projects/godot/emscripten/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/lto/crtbegin.o -lGL-mt-webgl2-ofb -lal -lhtml5 -lbulkmemory -lstubs-debug -lc-mt-debug -ldlmalloc-mt -lcompiler_rt-mt -lc++-mt-noexcept -lc++abi-debug-mt-noexcept -lsockets-mt -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr /tmp/tmpowj6vflhlibemscripten_js_symbols.so --import-memory --shared-memory --strip-debug --export-if-defined=main --export-if-defined=_emscripten_thread_init --export-if-defined=_emscripten_thread_exit --export-if-defined=_emscripten_thread_crashed --export-if-defined=_emscripten_tls_init --export-if-defined=pthread_self --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export-if-defined=__start_em_lib_deps --export-if-defined=__stop_em_lib_deps --export-if-defined=__start_em_js --export-if-defined=__stop_em_js --export-if-defined=__main_argc_argv --export-if-defined=fflush --export=emscripten_stack_get_end --export=emscripten_stack_get_free --export=emscripten_stack_get_base --export=emscripten_stack_get_current --export=emscripten_stack_init --export=stackSave --export=stackRestore --export=stackAlloc --export=__errno_location --export=_emscripten_thread_free_data --export=emscripten_main_runtime_thread_id --export=emscripten_main_thread_process_queued_calls --export=_emscripten_run_on_main_thread_js --export=emscripten_stack_set_limits --export=__get_temp_ret --export=__set_temp_ret --export=__funcs_on_exit --export=__wasm_call_ctors --export=_emscripten_thread_init --export=_emscripten_thread_exit --export-table -z stack-size=65536 --initial-memory=33554432 --max-memory=2147483648 --no-entry --stack-first' failed (returned 1)
scons: *** [bin/godot.web.template_release.wasm32.js] Error 1

Full build log: godot_build_web_lto_emscripten_3.1.45.log.zip

akien-mga commented 1 year ago

The issue is also present in Godot 4. Last working Emscripten version is also 3.1.36

I tried to confirm this and got different results, for me all versions up to 3.1.41 are working fine. The LTO linking step breaks from 3.1.42 onward, up to 3.1.45 (current latest). That seems consistent with upstream bug reports which report a regression in 3.1.42.

Did you test specifically that 3.1.37 and later weren't working? Are you sure it was the same error?

akien-mga commented 1 year ago

We can probably try to define -Wl,-u,scalbnf when we detect Emscripten >= 3.1.42, as described in https://github.com/emscripten-core/emscripten/issues/19781#issuecomment-1704365864.

I'll have a look at this after finalizing #80719 which refactors the compiler version detection, so this can be done cleanly.