Undefined exception symbols despite -fno-exceptions

[matsbror]

I have build wasi-sdk for risc-v to use on the VisionFive2 board from StarFive. I have issues with linking C++ programs as the linker insists on undefined references to __cxa_allocate_exceptionand __cxa_throw although I compiled using -fno-exceptions .

How do I resolve this?

$ /opt/wasi-sdk/bin/clang++ -v --target=wasm32-unknown-wasi --sysroot=/opt/wasi-sdk/share/wasi-sy
sroot -W -std=c++11 -fvisibility=hidden -DNDEBUG=1 -fno-exceptions -O2   src/HashSet.cpp -o hashset.wasm
clang version 16.0.0
Target: wasm32-unknown-wasi
Thread model: posix
InstalledDir: /opt/wasi-sdk/bin
 "/opt/wasi-sdk/bin/clang-16" -cc1 -triple wasm32-unknown-wasi -emit-obj -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name HashSet.cpp -mrelocation-model static -mframe-pointer=none -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -fcoverage-compilation-dir=/home/ubuntu/wasm/wabench/JetStream2/hashset -resource-dir /opt/wasi-sdk/lib/clang/16 -D NDEBUG=1 -isysroot /opt/wasi-sdk/share/wasi-sysroot -internal-isystem /opt/wasi-sdk/share/wasi-sysroot/include/wasm32-wasi/c++/v1 -internal-isystem /opt/wasi-sdk/share/wasi-sysroot/include/c++/v1 -internal-isystem /opt/wasi-sdk/lib/clang/16/include -internal-isystem /opt/wasi-sdk/share/wasi-sysroot/include/wasm32-wasi -internal-isystem /opt/wasi-sdk/share/wasi-sysroot/include -O2 -W -std=c++11 -fdeprecated-macro -fdebug-compilation-dir=/home/ubuntu/wasm/wabench/JetStream2/hashset -ferror-limit 19 -fvisibility=hidden -fgnuc-version=4.2.1 -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /tmp/HashSet-91e8a4.o -x c++ src/HashSet.cpp
clang -cc1 version 16.0.0 based upon LLVM 16.0.0 default target wasm32-wasi
ignoring nonexistent directory "/opt/wasi-sdk/share/wasi-sysroot/include/wasm32-wasi/c++/v1"
ignoring nonexistent directory "/opt/wasi-sdk/share/wasi-sysroot/include/wasm32-wasi"
#include "..." search starts here:
#include <...> search starts here:
 /opt/wasi-sdk/share/wasi-sysroot/include/c++/v1
 /opt/wasi-sdk/lib/clang/16/include
 /opt/wasi-sdk/share/wasi-sysroot/include
End of search list.
 "/opt/wasi-sdk/bin/wasm-ld" -m wasm32 -L/opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi /opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi/crt1-command.o /tmp/HashSet-91e8a4.o -lc++ -lc++abi -lc /opt/wasi-sdk/lib/clang/16/lib/wasi/libclang_rt.builtins-wasm32.a -o hashset.wasm
wasm-ld: error: /opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi/libc++abi.a(stdlib_new_delete.cpp.o): undefined symbol: __cxa_allocate_exception
wasm-ld: error: /opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi/libc++abi.a(stdlib_new_delete.cpp.o): undefined symbol: __cxa_throw
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

UPDATE: I get the same error with a wasi-sdk built for AARCH64

UPDATE2: I discovered that lbc++ is not compiled with -fno-exceptions . Will try to investigate why.

[matsbror]

I got a workaround working by setting LIBCXX_CXX_FLAGS in libcxx/CMakeLists.txt and LIBCXXABI_CXX_FLAGS in libcxxabi/CMakeLists.txt to -fno-exceptions.

But I still think this is an issue that someone more well versed in cmake intricacies could investigate.

[sbc100]

Yes, since wasi-sdk doesn't currently support exceptions we should probably be building libc++abi.a and libc++.a with -fno-exceptions. Would you like to send a patch?

[sbc100]

It looks like do already build with -DLIBCXXABI_ENABLE_EXCEPTIONS=OFF and -DLIBCXX_ENABLE_EXCEPTIONS=OFF, but I guess that is somehow not the same as -fno-exceptions?

[sbc100]

Which version of wasi-sdk are you using? Looking at the wasi-sdk-20 I don't see any references to __cxa_allocate_exception in libc++-abi.a:

$ llvm-nm wasi-sdk-20.0/share/wasi-sysroot/lib/wasm32-wasi/libc++abi.a | grep __cxa_allocate_exception

[matsbror]

/

Which version of wasi-sdk are you using? Looking at the wasi-sdk-20 I don't see any references to __cxa_allocate_exception in libc++-abi.a:

$ llvm-nm wasi-sdk-20.0/share/wasi-sysroot/lib/wasm32-wasi/libc++abi.a | grep __cxa_allocate_exception

I'm on 20, the latest release. Indeed, when I compile on x86_64, there are no references to __cxa_allocate_exception in either licxx.a or libcxxabi.a. However, when I compile on Ubuntu/RISC-V there are. To be fair, I compile on a RISCV docker container which might not behave correctly. The VisionFive2 board has only 4GB memory which was a bit low to build LLVM.

[matsbror]

It seems as the behaviour is the same on at least some ARM system. The following is from the log building on an Nvidia Jetson board:

[901/935] /home/jetson/src/wasi-sdk/build/install/opt/wasi-sdk/bin/clang++ --target=wasm32-wasi 
--sysroot=/home/jetson/src/wasi-sdk/build/install/opt/wasi-sdk/share/wasi-sysroot -DLIBCXX_BUILDING_LIBCXXABI 
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LIBCPP_BUILDING_LIBRARY 
-D_LIBCPP_DISABLE_NEW_DELETE_DEFINITIONS -D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER 
-D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS 
-I/home/jetson/src/wasi-sdk/src/llvm-project/libcxx/src -I/home/jetson/src/wasi-sdk/build/libcxx/include/c++/v1 
-I/home/jetson/src/wasi-sdk/src/llvm-project/libcxxabi/include -fdebug-prefix-map=/home/jetson/src/wasi-sdk=wasisdk://v20.7g3fb0057a6da0+m 
-Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual 
-Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -std=c++20 -nostdinc++ -MD -MT 
libcxx/src/CMakeFiles/cxx_static.dir/hash.cpp.o -MF libcxx/src/CMakeFiles/cxx_static.dir/hash.cpp.o.d -o 
libcxx/src/CMakeFiles/cxx_static.dir/hash.cpp.o -c /home/jetson/src/wasi-sdk/src/llvm-project/libcxx/src/hash.cpp

As you can see, there is no -fno-exception flag there which there is in the corresponding build on an x86 system.

[rajsite]

I have an existing wasi compliant wasm created with emscripten's standalone wasm mode + -fnoexceptions and trying to switch to wasi-sdk I'm running into the same missing symbols.

Using the wasi-sdk 20 release build on windows. The wasi-sdk change diff and build output.

The clang++ -v output:

wasi-sdk/bin/clang++ --sysroot=wasi-sdk/share/wasi-sysroot -std=c++14 -pedantic-errors -Wall -Wextra -Werror -Wno-long-long -fno-exceptions -DARMA_DONT_USE_LAPACK -DARMA_DONT_USE_NEWARP -DARMA_DONT_USE_ARPACK -DARMA_DONT_USE_BLAS -DARMA_DONT_USE_SUPERLU -DARMA_DONT_USE_HDF5 -DARMA_DONT_USE_OPENMP -DARMA_DONT_USE_STD_MUTEX -O3 -isystem imports/exprtk/ -isystem imports/armadillo-code/include/ -isystem imports/sigpack/sigpack/ -Isource/include source/main.cpp -lm -v -o dist/exorbitant.wasm
clang version 16.0.0
Target: wasm32-unknown-wasi
Thread model: posix
InstalledDir: D:/a/exorbitant/exorbitant/wasi-sdk/bin
 "D:/a/exorbitant/exorbitant/wasi-sdk/bin/clang++.exe" -cc1 -triple wasm32-unknown-wasi -emit-obj -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model static -mframe-pointer=none -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu generic -fvisibility=hidden -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -fcoverage-compilation-dir=D:/a/exorbitant/exorbitant -resource-dir D:/a/exorbitant/exorbitant/wasi-sdk/lib/clang/16 -isystem imports/exprtk/ -isystem imports/armadillo-code/include/ -isystem imports/sigpack/sigpack/ -D ARMA_DONT_USE_LAPACK -D ARMA_DONT_USE_NEWARP -D ARMA_DONT_USE_ARPACK -D ARMA_DONT_USE_BLAS -D ARMA_DONT_USE_SUPERLU -D ARMA_DONT_USE_HDF5 -D ARMA_DONT_USE_OPENMP -D ARMA_DONT_USE_STD_MUTEX -I source/include -isysroot wasi-sdk/share/wasi-sysroot -internal-isystem wasi-sdk/share/wasi-sysroot/include/wasm32-wasi/c++/v1 -internal-isystem wasi-sdk/share/wasi-sysroot/include/c++/v1 -internal-isystem D:/a/exorbitant/exorbitant/wasi-sdk/lib/clang/16/include -internal-isystem wasi-sdk/share/wasi-sysroot/include/wasm32-wasi -internal-isystem wasi-sdk/share/wasi-sysroot/include -O3 -Wall -Wextra -Werror -Wno-long-long -pedantic-errors -std=c++14 -fdeprecated-macro -fdebug-compilation-dir=D:/a/exorbitant/exorbitant -ferror-limit 19 -fgnuc-version=4.2.1 -vectorize-loops -vectorize-slp -o C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7[8](https://github.com/rajsite/exorbitant/actions/runs/5708505160/job/15466259810#step:8:9)45.o -x c++ source/main.cpp
clang -cc1 version [16](https://github.com/rajsite/exorbitant/actions/runs/5708505160/job/15466259810#step:8:17).0.0 based upon LLVM 16.0.0 default target wasm[32](https://github.com/rajsite/exorbitant/actions/runs/5708505160/job/15466259810#step:8:33)-wasi
ignoring nonexistent directory "wasi-sdk/share/wasi-sysroot/include/wasm32-wasi/c++/v1"
ignoring nonexistent directory "wasi-sdk/share/wasi-sysroot/include/wasm32-wasi"
#include "..." search starts here:
#include <...> search starts here:
 source/include
 imports/exprtk
 imports/armadillo-code/include
 imports/sigpack/sigpack
 wasi-sdk/share/wasi-sysroot/include/c++/v1
 D:/a/exorbitant/exorbitant/wasi-sdk/lib/clang/16/include
 wasi-sdk/share/wasi-sysroot/include
End of search list.
 "D:/a/exorbitant/exorbitant/wasi-sdk/bin/wasm-ld" -m wasm32 -Lwasi-sdk/share/wasi-sysroot/lib/wasm32-wasi wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi/crt1-command.o C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o -lm -lc++ -lc++abi -lc D:/a/exorbitant/exorbitant/wasi-sdk/lib/clang/16/lib/wasi/libclang_rt.builtins-wasm32.a -o dist/exorbitant.wasm
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_allocate_exception
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_throw
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_allocate_exception
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_throw
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_allocate_exception
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_throw
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_allocate_exception
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_throw
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_allocate_exception
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_throw
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_allocate_exception
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_throw
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_allocate_exception
wasm-ld: error: C:/Users/RUNNER~1/AppData/Local/Temp/main-dd7845.o: undefined symbol: __cxa_throw
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:47: dist/exorbitant.wasm] Error 1

Edit: Digging in emscripten a bit looks like they stub out the __cxa_throw calls with exceptions disabled.

So went for just stubbing out those as well and am able to build:

extern "C" {
   void __cxa_allocate_exception() {
      abort();
   }

   void __cxa_throw() {
      abort();
   }
}

[carlocab]

I encountered this too. In my case, I found that it happens whenever building libc++[abi].a without a WASM-compatible linker available on my system (i.e. wasm-ld or wasm-component-ld). I wasn't building the shared libraries, so I didn't think they would be needed.

I think that the reason this happens is that the libc++ build adds -fno-exceptions (or the platform equivalent) after testing that the compiler can be invoked successfully with those flags. See:

• https://github.com/llvm/llvm-project/blob/62ff85f0799560b42754ef77b5f64ca2c7feeff7/libcxx/CMakeLists.txt#L551-L560
• https://github.com/llvm/llvm-project/blob/62ff85f0799560b42754ef77b5f64ca2c7feeff7/runtimes/cmake/Modules/HandleFlags.cmake#L37-L40

However, if you have no WASM-compatible linker available, then CMake will (probably erroneously) conclude that your toolchain does not support -fno-exceptions and omit the flag entirely. You can see this from the output when I run CMake to configure my build:

-- Performing Test CXX_SUPPORTS_FNO_EXCEPTIONS_FLAG
System is unknown to cmake, create:
Platform/WASI to use this system, please post your config file on discourse.cmake.org so it can be added to cmake
-- Performing Test CXX_SUPPORTS_FNO_EXCEPTIONS_FLAG - Failed