Nightly fails on SH-4 cross compile

reneme commented 5 days ago

during RTL pass: ira
  src/lib/pubkey/classic_mceliece/cmce_keys_internal.cpp: In static member function ‘static std::shared_ptr<Botan::Classic_McEliece_PublicKeyInternal> Botan::Classic_McEliece_PublicKeyInternal::create_from_private_key(const Botan::Classic_McEliece_PrivateKeyInternal&)’:
  src/lib/pubkey/classic_mceliece/cmce_keys_internal.cpp:152:1: internal compiler error: in elimination_costs_in_insn, at reload1.cc:3536
    152 | }
        | ^
  0x16c6ddb internal_error(char const*, ...)
    ???:0
  0x64d6b8 fancy_abort(char const*, int, char const*)
    ???:0
  0xd10d61 calculate_elim_costs_all_insns()
    ???:0
  0xbb98de ira_costs()
  ccache sh4-linux-gnu-g++ -fPIC -fvisibility=hidden -fstack-protector -pthread -std=c++20 -D_REENTRANT  -O3 -Wall -Wextra -Wpedantic -Wstrict-aliasing -Wcast-align -Wmissing-declarations -Wpointer-arith -Wcast-qual -Wzero-as-null-pointer-constant -Wnon-virtual-dtor -Wold-style-cast -Wsuggest-override -Wshadow -Wextra-semi -Werror -Wno-error=strict-overflow -Wno-error=zero-as-null-pointer-constant -Wno-error=non-virtual-dtor -Wno-error=maybe-uninitialized -Wno-error=stringop-overread -Wno-error=stringop-overflow -Wno-error=free-nonheap-object -Wno-error=restrict -DBOTAN_IS_BEING_BUILT  -I build/include/public -I build/include/internal  -isystem build/include/external -c src/lib/pubkey/curve448/curve448_scalar.cpp -o build/obj/lib/pubkey_curve448_scalar.o
    ???:0
  0xbb294f ira_build()
    ???:0
  Please submit a full bug report, with preprocessed source (by using -freport-bug).
  Please include the complete backtrace with any bug report.

See: https://github.com/randombit/botan/actions/runs/12002657334/job/33454969797

FAlbertDev commented 4 days ago

The same error also seems to appear in some MSVC configurations:

Log of our CI

```log INFO: configure.py invoked with options "--cc=msvc --cpu=x86 --module-policy=bsi --enable-modules=tls12,tls13,pkcs11,xts,sha1,sha1_sse2,sha1_x86,eme_pkcs1,eme_raw,x509,emsa_pkcs1 --enable-shared" INFO: Configuring to build Botan 3.7.0 (revision git:ea47ac7ff7449630413690a5d0c089999bf7fbbe) INFO: Python version: "3.11.1 (tags/v3.11.1:a7a450f, Dec 6 2022, 19:58:39) [MSC v.1934 64 bit (AMD64)]" INFO: Autodetected platform information: OS="Windows" machine="AMD64" proc="Intel64 Family 6 Model 63 Stepping 0, GenuineIntel" INFO: Guessing target OS is windows (use --os to set) INFO: Canonicalized CPU target x86 to x86_32 INFO: Auto-detected compiler version msvc 19.34 INFO: Auto-detected compiler arch x86_32 INFO: Target is msvc:19.34-windows-x86_32 INFO: Assuming target x86_32 is little endian INFO: Skipping (incompatible CPU): aes_armv8 aes_power8 aes_vaes chacha_avx512 keccak_perm_bmi2 serpent_avx512 sha1_armv8 sha2_32_armv8 sha2_64_armv8 sha2_64_bmi2 shacal2_armv8 simd_avx512 sm4_armv8 sm4_gfni INFO: Skipping (incompatible OS): certstor_system_macos commoncrypto fd_unix getentropy INFO: Skipping (incompatible compiler): argon2_avx2 argon2_ssse3 chacha_avx2 serpent_avx2 sha2_32_bmi2 shacal2_avx2 simd_avx2 INFO: Skipping (not requested): adler32 aes_crystals_xof asio base32 base58 bcrypt_pbkdf blake2s certstor_flatfile certstor_sql certstor_sqlite3 certstor_system certstor_system_windows checksum compression crc24 crc32 cryptobox cshake_xof curve448 dilithium dilithium_aes dilithium_common dilithium_round3 dilithium_shake dlies ed448 ffi filters fpe_fe1 frodokem frodokem_aes frodokem_common hotp hss_lms jitter_rng kmac kyber kyber_90s kyber_common kyber_round3 ml_dsa ml_kem nist_keywrap par_hash passhash9 pbes2 pbkdf2 pcurves_brainpool256r1 pcurves_brainpool pcurves_brainpool512r1 pcurves_frp256v1 pcurves_impl pcurves_numsp512d1 pcurves_secp192r1 pcurves_secp224r1 pcurves_secp256k1 pcurves_secp256r1 pcurves_secp384r1 pcurves_secp521r1 pcurves_sm2p256v1 pgp_s2k pqcrystals psk_db raw_hash roughtime scrypt sessions_sql sessions_sqlite3 shake shake_cipher shake_xof slh_dsa_sha2 slh_dsa_shake sodium sphincsplus_common sphincsplus_sha2 sphincsplus_sha2_base sphincsplus_shake sphincsplus_shake_base srp6 thread_utils tls13_pqc tls_cbc tpm2_crypto_backend tpm2_ecc tpm2_rsa tree_hash tss uuid xof zfec zfec_sse2 zfec_vperm INFO: Skipping (prohibited by module policy): aria bcrypt blake2mac blowfish camellia cascade cast128 cfb chacha chacha20poly1305 chacha_avx2 chacha_rng chacha_simd32 comb4p des eax ed25519 elgamal emsa_raw emsa_x931 gost_28147 gost_3 gost_3411 idea idea_sse2 kdf1 kdf2 keccak kuznyechik lion mce md4 md5 noekeon noekeon_simd ocb ofb poly1305 prf_x9rc4 rfc6979 rmd160 salsa20 seed serpent serpent_avx2 serpent_simd shacal2 shacal2_armv8 shacal2_avx2 shacal2_simd shacal2_x86 siphash siv skein sm2 sm3 sm4 sm4_armv8 sp800_56a streebog threefish_512 twofish whirlpool x25519 x919_mac xmd INFO: Skipping (requires external dependency): boost bzip2 lzma sqlite3 tpm tpm2 zlib INFO: Loading modules: aead aes aes_ni aes_vperm argon2 argon2fmt asn1 auto_rng base base64 bigint blake2 block cbc ccm cmac cpuid ctr dh dl_algo dl_group dsa dyn_load ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa eme_oaep eme_pkcs1 eme_raw emsa_pkcs1 emsa_pssr entropy gcm ghash ghash_cpu ghash_vperm gmac hash hash_id hex hkdf hmac hmac_drbg http_util iso9796 kdf kdf1_iso18033 keccak_perm keypair locking_allocator mac mdx_hash mem_pool mgf1 mode_pad modes mp numbertheory pbkdf pcurves pem pk_pad pkcs11 poly_dbl prf_tls processor_rng pubkey rdseed rng rsa sha1 sha1_sse2 sha1_x86 sha2_32 sha2_32_x86 sha2_64 sha3 simd socket sp800_108 sp800_56c stateful_rng stream system_rng tls tls12 tls13 trunc_hash utils win32_stats x509 xmss xts INFO: Using copy to link files into build dir (use --link-method to change) INFO: Botan 3.7.0 (revision ...) (unreleased undated) build setup is complete cl /DBOTAN_DLL=__declspec(dllexport) /Zc:preprocessor /std:c++20 /EHs /GR /D_WIN32_WINNT=0x0600 /MD /bigobj /O2 /Oi /Zc:throwingNew /W4 /wd4251 /wd4275 /wd5072 -DBOTAN_IS_BEING_BUILT /I build/include/public /I build/include/internal /external:W0 /external:I build/include/external /nologo /c src/lib/asn1/alg_id.cpp /Fobuild/obj/lib/asn1_alg_id.obj ... cl /Zc:preprocessor /std:c++20 /EHs /GR /D_WIN32_WINNT=0x0600 /MD /bigobj /O2 /Oi /Zc:throwingNew /analyze /W4 /wd4251 /wd4275 /wd5072 -DBOTAN_IS_BEING_BUILT /I build/include/public /I build/include/internal /external:W0 /external:I build/include/external /nologo /c src/tests/test_utils_bitvector.cpp /Fobuild/obj/test/test_utils_bitvector.obj test_utils_bitvector.cpp C:\BuildTools\VC\Tools\MSVC\14.34.31933\include\type_traits(1724): fatal error C1001: Internal compiler error. (compiler file 'msc1.cpp', line 1592) To work around this problem, try simplifying or changing the program near the locations listed above. If possible please provide a repro here: https://developercommunity.visualstudio.com/ Please choose the Technical Support command on the Visual C++ Help menu, or open the Technical Support help file for more information C:\BuildTools\VC\Tools\MSVC\14.34.31933\include\type_traits(1724): note: see reference to alias template instantiation 'std::_Select_invoke_traits' being compiled build/include/internal\botan/internal/bitvector.h(76): note: see reference to alias template instantiation 'std::invoke_result_t' being compiled INTERNAL COMPILER ERROR in 'C:\BuildTools\VC\Tools\MSVC\14.34.31933\bin\HostX64\x64\cl.EXE' Please choose the Technical Support command on the Visual C++ Help menu, or open the Technical Support help file for more information NMAKE : fatal error U1077: 'C:\BuildTools\VC\Tools\MSVC\14.34.31933\bin\HostX64\x64\cl.EXE' : return code '0xc0000005' Stop. ```

We currently use MSVC 19.34. I have not yet tested if the error occurs with 19.42 in out setup (which we use in Botan). However, this looks like something we should investigate further,

FAlbertDev commented 4 days ago

Update: The bug MSVC produces seems unrelated to the one in SH-4. I reproduced the issue locally on Windows. Indeed, the bug only occurs in a previous version.

randombit commented 1 day ago

Fixed now via #4447

randombit / botan

Nightly fails on SH-4 cross compile #4444