randombit / botan

Cryptography Toolkit
https://botan.randombit.net
BSD 2-Clause "Simplified" License
2.55k stars 561 forks source link

Today's master fails to link on MacOS #2465

Closed mouse07410 closed 3 years ago

mouse07410 commented 3 years ago

macOS Catalina 10.15.7, Xcode-12.1, current master

. . . . .
cd . && ln -fs libbotan-3.0.dylib libbotan-3.0.0.0.dylib
clang++  -Xpreprocessor -fopenmp -fstack-protector -m64 -pthread -stdlib=libc++ -march=native -Os -Ofast -I/opt/local/include build/obj/cli/argon2.o build/obj/cli/asn1.o build/obj/cli/bcrypt.o build/obj/cli/cc_enc.o build/obj/cli/cli.o build/obj/cli/cli_rng.o build/obj/cli/codec.o build/obj/cli/compress.o build/obj/cli/encryption.o build/obj/cli/entropy.o build/obj/cli/hash.o build/obj/cli/hmac.o build/obj/cli/main.o build/obj/cli/math.o build/obj/cli/pbkdf.o build/obj/cli/pk_crypt.o build/obj/cli/psk.o build/obj/cli/pubkey.o build/obj/cli/roughtime.o build/obj/cli/sandbox.o build/obj/cli/speed.o build/obj/cli/timing_tests.o build/obj/cli/tls_client.o build/obj/cli/tls_http_server.o build/obj/cli/tls_proxy.o build/obj/cli/tls_server.o build/obj/cli/tls_utils.o build/obj/cli/tss.o build/obj/cli/utils.o build/obj/cli/x509.o  -L/opt/local/lib/libomp -lomp -L/opt/local/lib  -L. -lbotan-3 -L/opt/local/lib/libomp -lomp -L/opt/local/lib  -lboost_system-mt -lbz2 -lcrypto -ldl -llzma -lsqlite3 -lz -framework CoreFoundation -framework Security -o botan
Undefined symbols for architecture x86_64:
  "Botan::calendar_point::from_time_point(std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000l> > > const&)", referenced from:
      Botan_CLI::RoughtimeCheck::go() in roughtime.o
      Botan_CLI::Roughtime::query(std::__1::unique_ptr<Botan::Roughtime::Chain, std::__1::default_delete<Botan::Roughtime::Chain> >&, unsigned long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Botan::Ed25519_PublicKey const&) in roughtime.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Botan was configured this way:

./configure.py --prefix=/opt/local --with-os-features=security_framework,apple_keychain,commoncrypto,getentropy --with-openmp --with-openssl --with-boost --with-lzma --with-bzip2 --with-zlib --with-sqlite3 --with-python-version=2.7 --with-sphinx --with-pdf --cc-abi-flags='-march=native -Os -Ofast -I/opt/local/include'

  INFO: ./configure.py invoked with options "--prefix=/opt/local --with-os-features=security_framework,apple_keychain,commoncrypto,getentropy --with-openmp --with-openssl --with-boost --with-lzma --with-bzip2 --with-zlib --with-sqlite3 --with-python-version=2.7 --with-sphinx --with-pdf --cc-abi-flags=-march=native -Os -Ofast -I/opt/local/include"
   INFO: Configuring to build Botan 3.0.0-alpha0 (revision git:c56e3daa58da3cf8e0ed1b2b4b7ab7d30c516483)
   INFO: Running under 3.8.6 (default, Sep 28 2020, 04:41:02) [Clang 11.0.3 (clang-1103.0.32.62)]
   INFO: Implicit --cc-bin=clang++ due to environment variable CXX
   INFO: Implicit --cxxflags=-std=gnu++2a -O3 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk due to environment variable CXXFLAGS
   INFO: Autodetected platform information: OS="Darwin" machine="x86_64" proc="i386"
   INFO: Guessing target OS is darwin (use --os to set)
   INFO: Guessing target processor is a x86_64 (use --cpu to set)
   INFO: Using /etc/ssl/cert.pem as system certificate store
   INFO: Auto-detected compiler version 4.0
   INFO: Auto-detected compiler arch x86_64
   INFO: Target is clang:4.0-macos-x86_64
   INFO: Assuming target x86_64 is little endian
   INFO: Skipping (incompatible CPU): aes_armv8 aes_power8 sha1_armv8 sha2_32_armv8 sm4_armv8
   INFO: Skipping (incompatible OS): certstor_system_windows proc_walk win32_stats
   INFO: Skipping (requires external dependency): commoncrypto tpm
   INFO: Enabling use of external dependency boost
   INFO: Enabling use of external dependency bzip2
   INFO: Enabling use of external dependency lzma
   INFO: Enabling use of external dependency openssl
   INFO: Enabling use of external dependency sqlite3
   INFO: Enabling use of external dependency zlib
   INFO: Loading modules: adler32 aead aes aes_ni aes_vperm argon2 aria asio asn1 auto_rng base base32 base58 base64 bcrypt bcrypt_pbkdf bigint blake2 block blowfish boost bzip2 camellia cascade cast128 cbc ccm cecpq1 certstor_flatfile certstor_sql certstor_sqlite3 certstor_system certstor_system_macos cfb chacha chacha20poly1305 chacha_avx2 chacha_rng chacha_simd32 checksum cmac comb4p compression cpuid crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa dyn_load eax ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa1 emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm getentropy ghash ghash_cpu ghash_vperm gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp http_util idea idea_sse2 iso9796 kdf kdf1 kdf1_iso18033 kdf2 keccak keypair lion locking_allocator lzma mac mce md4 md5 mdx_hash mem_pool mgf1 mode_pad modes mp newhope nist_keywrap noekeon noekeon_simd numbertheory ocb ofb openssl par_hash passhash9 pbes2 pbkdf pbkdf2 pem pgp_s2k pk_pad pkcs11 poly1305 poly_dbl prf_tls prf_x942 processor_rng psk_db pubkey rc4 rdseed rfc3394 rfc6979 rmd160 rng roughtime rsa salsa20 scrypt seed serpent serpent_avx2 serpent_simd sessions_sql sessions_sqlite3 sha1 sha1_sse2 sha1_x86 sha2_32 sha2_32_bmi2 sha2_32_x86 sha2_64 sha2_64_bmi2 sha3 sha3_bmi2 shacal2 shacal2_avx2 shacal2_simd shacal2_x86 shake shake_cipher simd simd_avx2 siphash siv skein sm2 sm3 sm4 socket sodium sp800_108 sp800_56a sp800_56c sqlite3 srp6 stateful_rng stream streebog system_rng thread_utils threefish_512 threefish_512_avx2 tls tls_10 tls_cbc tss twofish utils uuid whirlpool x509 x919_mac xmss xts zlib
   INFO: Using symlink to link files into build dir (use --link-method to change)
   INFO: Botan 3.0.0-alpha0 (revision git:c56e3daa58da3cf8e0ed1b2b4b7ab7d30c516483) (unreleased undated) build setup is complete

Update

Commit 7c27982e27b9536 is the last one that builds successfully (and passes all the tests).

randombit commented 3 years ago

I suspect ae8dd59c3d4ed5e4b0e0319b470d8b019102bf11 fixes this though it's interesting that it worked before on both Linux and the macOS CI. Can you test ae8dd59c3d4ed5e4b0e0319b470d8b019102bf11 (since CI doesn't run into this I don't know for sure that it is resolved)

mouse07410 commented 3 years ago

No joy. Similar problem. Freshly-cloned Botan master, as there's no commit ae8dd59 anymore - replaced by #2467.

. . . . .
cd . && ln -fs libbotan-3.0.dylib libbotan-3.0.0.0.dylib
clang++  -Xpreprocessor -fopenmp -fstack-protector -m64 -pthread -stdlib=libc++ -march=native -O3 -I/opt/local/include build/obj/cli/argon2.o build/obj/cli/asn1.o build/obj/cli/bcrypt.o build/obj/cli/cc_enc.o build/obj/cli/cli.o build/obj/cli/cli_rng.o build/obj/cli/codec.o build/obj/cli/compress.o build/obj/cli/encryption.o build/obj/cli/entropy.o build/obj/cli/hash.o build/obj/cli/hmac.o build/obj/cli/main.o build/obj/cli/math.o build/obj/cli/pbkdf.o build/obj/cli/pk_crypt.o build/obj/cli/psk.o build/obj/cli/pubkey.o build/obj/cli/roughtime.o build/obj/cli/sandbox.o build/obj/cli/speed.o build/obj/cli/timing_tests.o build/obj/cli/tls_client.o build/obj/cli/tls_http_server.o build/obj/cli/tls_proxy.o build/obj/cli/tls_server.o build/obj/cli/tls_utils.o build/obj/cli/tss.o build/obj/cli/utils.o build/obj/cli/x509.o  -L/opt/local/lib/libomp -lomp -L/opt/local/lib  -L. -lbotan-3 -L/opt/local/lib/libomp -lomp -L/opt/local/lib  -lboost_system-mt -lbz2 -lcrypto -ldl -llzma -lsqlite3 -lz -framework CoreFoundation -framework Security -o botan
Undefined symbols for architecture x86_64:
  "Botan::calendar_point::calendar_point(std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000l> > > const&)", referenced from:
      Botan_CLI::RoughtimeCheck::go() in roughtime.o
      Botan_CLI::Roughtime::query(std::__1::unique_ptr<Botan::Roughtime::Chain, std::__1::default_delete<Botan::Roughtime::Chain> >&, unsigned long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Botan::Ed25519_PublicKey const&) in roughtime.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:72: botan] Error 1

Config:

  INFO: ./configure.py invoked with options "--prefix=/opt/local --with-os-features=security_framework,apple_keychain,commoncrypto,getentropy --with-openmp --with-openssl --with-boost --with-lzma --with-bzip2 --with-zlib --with-sqlite3 --with-python-version=2.7 --with-sphinx --with-pdf --cc-abi-flags=-march=native -O3 -I/opt/local/include"
   INFO: Configuring to build Botan 3.0.0-alpha0 (revision git:45f1a79bcd051ea8eeda66b96edf3f7163e38b4b)
   INFO: Running under 3.8.6 (default, Sep 28 2020, 04:41:02) [Clang 11.0.3 (clang-1103.0.32.62)]
   INFO: Implicit --cc-bin=clang++ due to environment variable CXX
   INFO: Implicit --cxxflags=-std=gnu++17 -O3 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk due to environment variable CXXFLAGS
   INFO: Autodetected platform information: OS="Darwin" machine="x86_64" proc="i386"
   INFO: Guessing target OS is darwin (use --os to set)
   INFO: Guessing target processor is a x86_64 (use --cpu to set)
   INFO: Using /etc/ssl/cert.pem as system certificate store
   INFO: Auto-detected compiler version 4.0
   INFO: Auto-detected compiler arch x86_64
   INFO: Target is clang:4.0-macos-x86_64
   INFO: Assuming target x86_64 is little endian
   INFO: Skipping (incompatible CPU): aes_armv8 aes_power8 sha1_armv8 sha2_32_armv8 sm4_armv8
   INFO: Skipping (incompatible OS): certstor_system_windows proc_walk win32_stats
   INFO: Skipping (requires external dependency): commoncrypto tpm
   INFO: Enabling use of external dependency boost
   INFO: Enabling use of external dependency bzip2
   INFO: Enabling use of external dependency lzma
   INFO: Enabling use of external dependency openssl
   INFO: Enabling use of external dependency sqlite3
   INFO: Enabling use of external dependency zlib
   INFO: Loading modules: adler32 aead aes aes_ni aes_vperm argon2 aria asio asn1 auto_rng base base32 base58 base64 bcrypt bcrypt_pbkdf bigint blake2 block blowfish boost bzip2 camellia cascade cast128 cbc ccm cecpq1 certstor_flatfile certstor_sql certstor_sqlite3 certstor_system certstor_system_macos cfb chacha chacha20poly1305 chacha_avx2 chacha_rng chacha_simd32 checksum cmac comb4p compression cpuid crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa dyn_load eax ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa1 emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm getentropy ghash ghash_cpu ghash_vperm gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp http_util idea idea_sse2 iso9796 kdf kdf1 kdf1_iso18033 kdf2 keccak keypair lion locking_allocator lzma mac mce md4 md5 mdx_hash mem_pool mgf1 mode_pad modes mp newhope nist_keywrap noekeon noekeon_simd numbertheory ocb ofb openssl par_hash passhash9 pbes2 pbkdf pbkdf2 pem pgp_s2k pk_pad pkcs11 poly1305 poly_dbl prf_tls prf_x942 processor_rng psk_db pubkey rc4 rdseed rfc3394 rfc6979 rmd160 rng roughtime rsa salsa20 scrypt seed serpent serpent_avx2 serpent_simd sessions_sql sessions_sqlite3 sha1 sha1_sse2 sha1_x86 sha2_32 sha2_32_bmi2 sha2_32_x86 sha2_64 sha2_64_bmi2 sha3 sha3_bmi2 shacal2 shacal2_avx2 shacal2_simd shacal2_x86 shake shake_cipher simd simd_avx2 siphash siv skein sm2 sm3 sm4 socket sodium sp800_108 sp800_56a sp800_56c sqlite3 srp6 stateful_rng stream streebog system_rng thread_utils threefish_512 threefish_512_avx2 tls tls_10 tls_cbc tss twofish utils uuid whirlpool x509 x919_mac xmss xts zlib
   INFO: Using symlink to link files into build dir (use --link-method to change)
   INFO: Botan 3.0.0-alpha0 (revision git:45f1a79bcd051ea8eeda66b96edf3f7163e38b4b) (unreleased undated) build setup is complete

Complete make output: make-out.txt

randombit commented 3 years ago

I am somewhat at a loss.

If you add -k to make, do the tests link? (I'm guessing not but if that works it might be a clue as to what is happening)

mouse07410 commented 3 years ago

Darn. The current master works. Another complete refresh (re-clone and such), another re-build, and it links (and passes the tests).

Darn... No clue, but at least it works...

Thanks for the efforts!