search

mosra / magnum

Lightweight and modular C++11 graphics middleware for games and data visualization
https://magnum.graphics/
Other
4.79k stars 440 forks source link

Building Magnum on Emscripten 2.0.17 to 2.0.24 fails with undefined symbol: malloc #507

Closed werto87 closed 2 years ago

werto87 commented 3 years ago

Hi, when I compile magnum to wasm i get the error "Could NOT find Corrade (missing: Utility)". I setup the toolchain file ("set(EMSCRIPTEN_PREFIX "/usr/lib/emscripten")") and build corrade for wasm. Then I execute "makepkg -fp PKGBUILD-emscripten-wasm" in "/magnum/package/archlinux" and get the output: ==> Making package: emscripten-magnum dev.wasm-1 (2021-05-26T00:44:21 CEST) ==> Checking runtime dependencies... ==> Checking buildtime dependencies... ==> Retrieving sources... ==> Extracting sources... ==> Removing existing $pkgdir/ directory... ==> Starting build()... -- The CXX compiler identification is Clang 13.0.0 -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/lib/emscripten/em++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done CMake Error at /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message): Could NOT find Corrade (missing: Utility) Call Stack (most recent call first): /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE) modules/FindCorrade.cmake:605 (find_package_handle_standard_args) CMakeLists.txt:51 (find_package)

Configuring incomplete, errors occurred! See also "/home/walde/aur/magnum/build-emscripten-wasm/CMakeFiles/CMakeOutput.log". See also "/home/walde/aur/magnum/build-emscripten-wasm/CMakeFiles/CMakeError.log". ==> ERROR: A failure occurred in build(). Aborting...

/usr/lib/emscripten/system/lib" contains: libCorradeInterconnect.so, libCorradePluginManager.so, libCorradeTestSuite.so, libCorradeUtility.so

mosra commented 3 years ago

Hi!

Corrade was built and installed using the PKGBUILD-emscripten-wasm in corrade as well, using the same toolchain file? It's strange that it installed *.sos, it should be generating static libraries with the *.bc extension.

Magnum (and the toolchain both Corrade and Magnum use) then looks for *.bc files and because there are only *.sos, it fails to find them.

werto87 commented 3 years ago

Hi, thank you for the clues. I will try a fresh start and will report what kind of files I get.

werto87 commented 3 years ago

I got the .bc files.

when running "sudo pacman -U emscripten-corrade-dev.wasm-1-any.pkg.tar.zst" i get some warnings: warning: could not get file information for usr/lib/emscripten/system/lib/libCorradeInterconnect.so warning: could not get file information for usr/lib/emscripten/system/lib/libCorradePluginManager.so warning: could not get file information for usr/lib/emscripten/system/lib/libCorradeTestSuite.so warning: could not get file information for usr/lib/emscripten/system/lib/libCorradeUtility.so

I will try to build magnum now

mosra commented 3 years ago

Looks like some stale files from a manual install? :)

To ensure you have a clean setup, I'd suggest manually removing the whole /usr/lib/emscripten and installing emscripten + emscripten-corrade again :)

werto87 commented 3 years ago

magnum finds corrade now

-- Found Corrade: /usr/lib/emscripten/system/include found components: Containers rc Utility -- Found Corrade: /usr/lib/emscripten/system/include found components: Containers rc Utility PluginManager -- Found Corrade: /usr/lib/emscripten/system/include found components: Containers rc Utility Main TestSuite -- Found Corrade: /usr/lib/emscripten/system/include found components: Containers rc Utility PluginManager

I think the problem was that I had some .so files in the emscripten/system folder.

werto87 commented 3 years ago

Looks like some stale files from a manual install? :)

To ensure you have a clean setup, I'd suggest manually removing the whole /usr/lib/emscripten and installing emscripten + emscripten-corrade again :)

Yeah I was fighting hard :)

werto87 commented 3 years ago

I am getting another error

[walde@localhost archlinux]$ makepkg --nocheck -fp PKGBUILD-emscripten-wasm ==> Making package: emscripten-magnum dev.wasm-1 (2021-05-26T16:19:06 CEST) ==> Checking runtime dependencies... ==> Checking buildtime dependencies... ==> Retrieving sources... ==> Extracting sources... ==> Removing existing $pkgdir/ directory... ==> Starting build()... -- Found Corrade: /usr/lib/emscripten/system/include found components: Containers rc Utility -- Found Corrade: /usr/lib/emscripten/system/include found components: Containers rc Utility PluginManager -- Found Corrade: /usr/lib/emscripten/system/include found components: Containers rc Utility Main TestSuite -- Found Corrade: /usr/lib/emscripten/system/include found components: Containers rc Utility PluginManager -- Configuring done -- Generating done -- Build files have been written to: /home/walde/aur/magnum/build-emscripten-wasm [7/119] Linking CXX executable Release/bin/magnum-gl-info.js FAILED: Release/bin/magnum-gl-info.js : && /usr/lib/emscripten/em++ -s WASM=1 -Wno-warn-absolute-paths -DNDEBUG -O3 -O3 -DNDEBUG -s WASM=1 -O3 --llvm-lto 1 src/Magnum/Platform/CMakeFiles/magnum-gl-info.dir/gl-info.cpp.o -o Release/bin/magnum-gl-info.js Release/lib/MagnumGL.bc Release/lib/MagnumWindowlessEglApplication.bc Release/lib/MagnumGL.bc Release/lib/Magnum.bc /usr/lib/emscripten/system/lib/CorradeUtility.bc /usr/lib/emscripten/system/lib/CorradeUtility.bc -lGL && cd /home/walde/aur/magnum/build-emscripten-wasm/src/Magnum/Platform && /usr/bin/cmake -E copy_if_different /home/walde/aur/magnum/src/Magnum/Platform/WebApplication.css /home/walde/aur/magnum/src/Magnum/Platform/WindowlessEmscriptenApplication.js /home/walde/aur/magnum/build-emscripten-wasm/Release/bin && /usr/bin/cmake -E copy_if_different /home/walde/aur/magnum/src/Magnum/Platform/gl-info.html /home/walde/aur/magnum/build-emscripten-wasm/Release/bin/magnum-gl-info.html emcc:WARNING: --llvm-lto ignored when using llvm backend error: undefined symbol: malloc (referenced by $stringToNewUTF8deps: ['malloc'], referenced by glGetStringdeps: ['$stringToNewUTF8','$GL'], referenced by top-level compiled C/C++ code) warning: Link with -s LLD_REPORT_UNDEFINED to get more information on undefined symbols warning: To disable errors for undefined symbols use -s ERROR_ON_UNDEFINED_SYMBOLS=0 warning: _malloc may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library Error: Aborting compilation due to previous errors em++: error: '/usr/bin/node /usr/lib/emscripten/src/compiler.js /tmp/tmpecl8dtvf.txt' failed (1) [14/119] Building CXX object src/Magnum/SceneGr...eFiles/MagnumSceneGraph.dir/instantiation.cpp.o ninja: build stopped: subcommand failed. ==> ERROR: A failure occurred in build(). Aborting...

mosra commented 3 years ago

Looks like some fancy new emscripten breakage (this happens all the time, unfortunately). What version are you on? Locally I'm on 2.0.10 and it works here -- can you try downgrading? Get a package from ArchLinux Archive, for example.

I'll have to fix this of course, but a downgrade could be a temporary solution to unblock you.

werto87 commented 3 years ago

I am on 2.0.17. I will try 2.0.10

werto87 commented 3 years ago

I was not able to get emscripten 2.0.10 running from arch aur. I will try to use emsdk to get to version 2.0.10.

werto87 commented 3 years ago

I was able to build magnum with emscripten 2.0.10 from emsdk. Note: if you are on arch linux and want to build magnum for wasm and are using emsdk you need to remove the dependency 'emscripten' from the PKGBUILD-emscripten-wasm file. Thank you @mosra for your help.

mosra commented 3 years ago

Keeping this open as a reminder to fix the build issue ;)

FYI, here's older Emscripten versions (this is what I'm always doing, in case I upgrade and get a breakage and I have nothing in /var/cache/pacman/pkg anymore, I get it from here): https://archive.archlinux.org/packages/e/emscripten/

mosra commented 2 years ago

Even a year later I still don't have any better solution than just ignoring affected Emscripten versions -- after all, it's malloc() being a dependency of Emscripten's own glGetString(), which I doubt I should be explicitly accounting for as an end user. (If it would be due to my own JS code inside EM_ASM() causing this, then that'd be a different scenario, but it isn't, it's glGetString().)

In my testing this started happening in 2.0.14 (probably due to https://github.com/emscripten-core/emscripten/pull/13434), happened still in 2.0.24 and then mysteriously went away in 2.0.25. I went through the commit log between the two versions and found nothing even vaguely related, so I simply don't know.

The only way to fix this on 2.0.14 to 2.0.24 is to supply -sEXPORTED_RUNTIME_METHODS=malloc -sEXPORTED_FUNCTIONS=_malloc,_main -sDEFAULT_LIBRARY_FUNCS_TO_INCLUDE=malloc to the linker (or maybe some subset of these three, didn't try further). But these Emscripten options always specify all exported functions, which is the shittiest possible way to implement such an option -- i.e., if -sEXPORTED_FUNCTIONS=callMyFooThing is passed from somewhere else as well, depending on the order at which they appear on the linker command line, either just callMyFooThing or just _main and _malloc gets exported. Thus I can't possibly pass this flag from the library CMake files to fix the build, to avoid breaking user builds that specify the same option, the end user is forced to do that instead if required.