randombit / botan

Cryptography Toolkit
https://botan.randombit.net
BSD 2-Clause "Simplified" License
2.59k stars 570 forks source link

MacOS Sonoma: cannot link using Clang-19 #4442

Open mouse07410 opened 1 day ago

mouse07410 commented 1 day ago

Apple Silicon M2 Mac, running MacOS Sonoma 14.7.1. Xcode-16.1, Macports-installed Clang-19.

Build of Botan-3 master fails at the linking stage:

clang++ -fstack-protector -pthread -stdlib=libc++ build/obj/test/main.o build/obj/test/runner_test_reporter.o build/obj/test/runner_test_runner.o build/obj/test/runner_test_stdout_reporter.o build/obj/test/runner_test_xml_reporter.o build/obj/test/test_aead.o build/obj/test/test_alt_name.o build/obj/test/test_asn1.o build/obj/test/test_bigint.o build/obj/test/test_block.o build/obj/test/test_blowfish.o build/obj/test/test_bufcomp.o build/obj/test/test_certstor.o build/obj/test/test_certstor_flatfile.o build/obj/test/test_certstor_system.o build/obj/test/test_certstor_utils.o build/obj/test/test_cmce.o build/obj/test/test_codec.o build/obj/test/test_compression.o build/obj/test/test_cryptobox.o build/obj/test/test_crystals.o build/obj/test/test_ct_utils.o build/obj/test/test_dh.o build/obj/test/test_dilithium.o build/obj/test/test_dl_group.o build/obj/test/test_dlies.o build/obj/test/test_dsa.o build/obj/test/test_ec_group.o build/obj/test/test_ecc_h2c.o build/obj/test/test_ecc_pointmul.o build/obj/test/test_ecdh.o build/obj/test/test_ecdsa.o build/obj/test/test_ecgdsa.o build/obj/test/test_ecies.o build/obj/test/test_eckcdsa.o build/obj/test/test_ed25519.o build/obj/test/test_ed448.o build/obj/test/test_elgamal.o build/obj/test/test_entropy.o build/obj/test/test_ffi.o build/obj/test/test_filters.o build/obj/test/test_fpe.o build/obj/test/test_frodokem.o build/obj/test/test_gf2m.o build/obj/test/test_gost_3410.o build/obj/test/test_hash.o build/obj/test/test_hash_id.o build/obj/test/test_hss_lms.o build/obj/test/test_jitter_rng.o build/obj/test/test_kdf.o build/obj/test/test_keccak_helpers.o build/obj/test/test_keywrap.o build/obj/test/test_kyber.o build/obj/test/test_lmots.o build/obj/test/test_lms.o build/obj/test/test_mac.o build/obj/test/test_mceliece.o build/obj/test/test_modes.o build/obj/test/test_mp.o build/obj/test/test_name_constraint.o build/obj/test/test_ocb.o build/obj/test/test_ocsp.o build/obj/test/test_octetstring.o build/obj/test/test_oid.o build/obj/test/test_os_utils.o build/obj/test/test_otp.o build/obj/test/test_pad.o build/obj/test/test_passhash.o build/obj/test/test_pbkdf.o build/obj/test/test_pcurves.o build/obj/test/test_pem.o build/obj/test/test_pk_pad.o build/obj/test/test_pkcs11_high_level.o build/obj/test/test_pkcs11_low_level.o build/obj/test/test_psk_db.o build/obj/test/test_pubkey.o build/obj/test/test_rfc6979.o build/obj/test/test_rng_behavior.o build/obj/test/test_rng_kat.o build/obj/test/test_rngs.o build/obj/test/test_roughtime.o build/obj/test/test_rsa.o build/obj/test/test_simd.o build/obj/test/test_siv.o build/obj/test/test_sm2.o build/obj/test/test_sodium.o build/obj/test/test_sphincsplus.o build/obj/test/test_sphincsplus_fors.o build/obj/test/test_sphincsplus_utils.o build/obj/test/test_sphincsplus_wots.o build/obj/test/test_srp6.o build/obj/test/test_stream.o build/obj/test/test_strong_type.o build/obj/test/test_tests.o build/obj/test/test_thread_utils.o build/obj/test/test_tls.o build/obj/test/test_tls_cipher_state.o build/obj/test/test_tls_handshake_layer_13.o build/obj/test/test_tls_handshake_state_13.o build/obj/test/test_tls_handshake_transitions.o build/obj/test/test_tls_hybrid_kem_key.o build/obj/test/test_tls_messages.o build/obj/test/test_tls_record_layer_13.o build/obj/test/test_tls_rfc8448.o build/obj/test/test_tls_session_manager.o build/obj/test/test_tls_signature_scheme.o build/obj/test/test_tls_stream_integration.o build/obj/test/test_tls_transcript_hash_13.o build/obj/test/test_tpm.o build/obj/test/test_tpm2.o build/obj/test/test_tss.o build/obj/test/test_uri.o build/obj/test/test_utils.o build/obj/test/test_utils_bitvector.o build/obj/test/test_utils_buffer.o build/obj/test/test_workfactor.o build/obj/test/test_x25519.o build/obj/test/test_x448.o build/obj/test/test_x509_dn.o build/obj/test/test_x509_path.o build/obj/test/test_xmss.o build/obj/test/test_xof.o build/obj/test/test_zfec.o build/obj/test/tests.o build/obj/test/unit_asio_stream.o build/obj/test/unit_ecdsa.o build/obj/test/unit_tls.o build/obj/test/unit_tls_policy.o build/obj/test/unit_x509.o -L.  -std=c++20 -stdlib=libc++ -O3 -I/opt/local/libexec/boost/1.81/include -I/opt/local/include  -L/opt/local/lib/libomp -lomp -L/opt/local/libexec/boost/1.81/lib  -lbotan-3 -L/opt/local/lib/libomp -lomp -L/opt/local/libexec/boost/1.81/lib   -lbz2 -ldl -llzma -lsqlite3 -lz -framework CoreFoundation -framework Security  -o botan-test3
ld: warning: ignoring duplicate libraries: '-lomp'
dyld[21687]: Symbol not found: __ZNSt13exception_ptr31__from_native_exception_pointerEPv
  Referenced from: <7D19A0A3-EFD4-319C-9CE9-95397405717B> /Users/ur20980/src/botan/botan-test3
  Expected in:     <C4E10D69-6F13-32B1-831D-82DA32D7DB88> /usr/lib/libc++.1.dylib

Build succeeds and all the tests pass when using Xcode Clang.

Here's how I configure the build:

./configure.py --program-suffix=3 --prefix=/opt/local --with-os-features=security_framework,apple_keychain,commoncrypto,getentropy --with-commoncrypto --with-boost --with-lzma --with-bzip2 --with-zlib --with-sqlite3 --with-python-version=3.12 --with-sphinx --with-pdf --system-cert-bundle=/opt/local/share/curl/curl-ca-bundle.crt 2>&1 | tee conf-out.txt
reneme commented 20 hours ago

Could it be that you're accidentally linking to a pre clang-19 version of libc++? I'm assuming that, because the symbol is expected in /usr/lib/libc++.1.dylib which (I guess) is the system-provided one. I'm assuming you got clang-19 from homebrew, therefore I'd expect the latest libc++-19 to be somewhere in /usr/local.

Didn't read into all the details, but this might help: https://github.com/llvm/llvm-project/issues/84392

mouse07410 commented 8 hours ago

Thank you - that's useful!

I tried to follow, but apparently what I did was not sufficient:

'/opt/local/bin/clang++-mp-19' -fstack-protector -m64 -pthread -stdlib=libc++ -march=native build/obj/test/main.o build/obj/test/runner_test_reporter.o build/obj/test/runner_test_runner.o build/obj/test/runner_test_stdout_reporter.o build/obj/test/runner_test_xml_reporter.o build/obj/test/test_aead.o build/obj/test/test_alt_name.o build/obj/test/test_asn1.o build/obj/test/test_bigint.o build/obj/test/test_block.o build/obj/test/test_blowfish.o build/obj/test/test_bufcomp.o build/obj/test/test_certstor.o build/obj/test/test_certstor_flatfile.o build/obj/test/test_certstor_system.o build/obj/test/test_certstor_utils.o build/obj/test/test_cmce.o build/obj/test/test_codec.o build/obj/test/test_compression.o build/obj/test/test_cryptobox.o build/obj/test/test_crystals.o build/obj/test/test_ct_utils.o build/obj/test/test_dh.o build/obj/test/test_dilithium.o build/obj/test/test_dl_group.o build/obj/test/test_dlies.o build/obj/test/test_dsa.o build/obj/test/test_ec_group.o build/obj/test/test_ecc_h2c.o build/obj/test/test_ecc_pointmul.o build/obj/test/test_ecdh.o build/obj/test/test_ecdsa.o build/obj/test/test_ecgdsa.o build/obj/test/test_ecies.o build/obj/test/test_eckcdsa.o build/obj/test/test_ed25519.o build/obj/test/test_ed448.o build/obj/test/test_elgamal.o build/obj/test/test_entropy.o build/obj/test/test_ffi.o build/obj/test/test_filters.o build/obj/test/test_fpe.o build/obj/test/test_frodokem.o build/obj/test/test_gf2m.o build/obj/test/test_gost_3410.o build/obj/test/test_hash.o build/obj/test/test_hash_id.o build/obj/test/test_hss_lms.o build/obj/test/test_jitter_rng.o build/obj/test/test_kdf.o build/obj/test/test_keccak_helpers.o build/obj/test/test_keywrap.o build/obj/test/test_kyber.o build/obj/test/test_lmots.o build/obj/test/test_lms.o build/obj/test/test_mac.o build/obj/test/test_mceliece.o build/obj/test/test_modes.o build/obj/test/test_mp.o build/obj/test/test_name_constraint.o build/obj/test/test_ocb.o build/obj/test/test_ocsp.o build/obj/test/test_octetstring.o build/obj/test/test_oid.o build/obj/test/test_os_utils.o build/obj/test/test_otp.o build/obj/test/test_pad.o build/obj/test/test_passhash.o build/obj/test/test_pbkdf.o build/obj/test/test_pcurves.o build/obj/test/test_pem.o build/obj/test/test_pk_pad.o build/obj/test/test_pkcs11_high_level.o build/obj/test/test_pkcs11_low_level.o build/obj/test/test_psk_db.o build/obj/test/test_pubkey.o build/obj/test/test_rfc6979.o build/obj/test/test_rng_behavior.o build/obj/test/test_rng_kat.o build/obj/test/test_rngs.o build/obj/test/test_roughtime.o build/obj/test/test_rsa.o build/obj/test/test_simd.o build/obj/test/test_siv.o build/obj/test/test_sm2.o build/obj/test/test_sodium.o build/obj/test/test_sphincsplus.o build/obj/test/test_sphincsplus_fors.o build/obj/test/test_sphincsplus_utils.o build/obj/test/test_sphincsplus_wots.o build/obj/test/test_srp6.o build/obj/test/test_stream.o build/obj/test/test_strong_type.o build/obj/test/test_tests.o build/obj/test/test_thread_utils.o build/obj/test/test_tls.o build/obj/test/test_tls_cipher_state.o build/obj/test/test_tls_handshake_layer_13.o build/obj/test/test_tls_handshake_state_13.o build/obj/test/test_tls_handshake_transitions.o build/obj/test/test_tls_hybrid_kem_key.o build/obj/test/test_tls_messages.o build/obj/test/test_tls_record_layer_13.o build/obj/test/test_tls_rfc8448.o build/obj/test/test_tls_session_manager.o build/obj/test/test_tls_signature_scheme.o build/obj/test/test_tls_stream_integration.o build/obj/test/test_tls_transcript_hash_13.o build/obj/test/test_tpm.o build/obj/test/test_tpm2.o build/obj/test/test_tss.o build/obj/test/test_uri.o build/obj/test/test_utils.o build/obj/test/test_utils_bitvector.o build/obj/test/test_utils_buffer.o build/obj/test/test_workfactor.o build/obj/test/test_x25519.o build/obj/test/test_x448.o build/obj/test/test_x509_dn.o build/obj/test/test_x509_path.o build/obj/test/test_xmss.o build/obj/test/test_xof.o build/obj/test/test_zfec.o build/obj/test/tests.o build/obj/test/unit_asio_stream.o build/obj/test/unit_ecdsa.o build/obj/test/unit_tls.o build/obj/test/unit_tls_policy.o build/obj/test/unit_x509.o -L.  '/opt/local/libexec/llvm-19/lib/libc++/libc++.dylib' -std=c++20 -stdlib=libc++ -O3  -I/opt/local/libexec/boost/1.81/include -I/opt/local/include   -lbotan-3 -lbz2 -ldl -llzma -lsqlite3 -lz -framework CoreFoundation -framework Security  -o botan-test3
dyld[40515]: Library not loaded: @rpath/libc++.1.dylib
  Referenced from: <E90980F6-7B6D-3119-A09E-908BE4102120> /Users/ur20980/src/botan/botan-test3
  Reason: tried: '/libc++.1.dylib' (no such file), './libc++.1.dylib' (no such file)