make requires explicit CXXFLAGS="-m32" and LDFLAGS="-m32" when building in 32 bit mode on a 64 bit machine (gcc)

[DevSidious]

Shouldn't configure.py automatically set CXXFLAGS="-m32" and LDFLAGS="-m32" when invoked with explicit --cpu=x86_32 (suggesting the intention to build in 32 bit mode on a 64 bit machine)?

SCENARIO:

INFO: ./configure.py invoked with options "--cpu=x86_32 --minimized-build --enable-modules=x509,ecdsa,rsa,emsa_pkcs1,sha2_64,aes --disable-shared-library --debug-mode --amalgamation --single-amalgamation-file --prefix=/opt/dev/botan/2.5.0/debug"
INFO: Guessing target OS is linux (use --os to set)
INFO: Guessing to use compiler gcc (use --cc or CXX to set)
INFO: Auto-detected compiler version 5.4
INFO: Target is gcc:5.4-linux-x86_32
INFO: Skipping (incompatible OS): darwin_secrandom getentropy win32_stats
INFO: Skipping (incompatible compiler): aes_armv8 pmull sha1_armv8 sha2_32_armv8
INFO: Skipping (not requested): adler32 aead aes_ni aes_ssse3 aont aria auto_rng bcrypt blake2 blowfish camellia cascade cast cbc cbc_mac ccm cecpq1 certstor_sql certstor_sqlite3 cfb chacha chacha20poly1305 chacha_rng chacha_sse2 checksum clmul clmul_ssse3 cmac codec_filt comb4p compression crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa dyn_load eax ecdh ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa_raw emsa_x931 fd_unix ffi filters fpe_fe1 gcm gmac gost_28147 gost_3410 gost_3411 hkdf hmac hmac_drbg hotp http_util idea idea_sse2 iso9796 kasumi kdf1 kdf1_iso18033 kdf2 keccak lion locking_allocator mce mceies md4 md5 misty1 mode_pad modes newhope nist_keywrap noekeon noekeon_simd ocb ofb par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pgp_s2k pkcs11 poly1305 poly_dbl prf_tls prf_x942 proc_walk psk_db rc4 rdrand rdrand_rng rdseed rfc3394 rfc6979 rmd160 salsa20 seed serpent serpent_simd sessions_sql sessions_sqlite3 sha1_sse2 sha1_x86 sha2_32_x86 sha3 shacal2 shacal2_simd shacal2_x86 shake shake_cipher simd siphash siv skein sm2 sm3 sm4 socket sp800_108 sp800_56a sp800_56c srp6 stateful_rng stream streebog system_rng thread_utils threefish threefish_avx2 tiger tls tls_cbc tss twofish whirlpool x919_mac xmss xtea xts
INFO: Skipping (requires external dependency): bearssl boost bzip2 lzma openssl sqlite3 tpm zlib
INFO: Loading modules: aes asn1 base base64 bigint block cpuid ec_gfp ec_group ecc_key ecdsa emsa1 emsa_pkcs1 emsa_pssr entropy hash hash_id hex kdf keypair mac mdx_hash mgf1 mp numbertheory pem pk_pad pubkey rng rsa sha1 sha2_32 sha2_64 utils x509
INFO: Using symlink to link files into build dir (use --link-method to change)
INFO: Writing amalgamation header to botan_all.h
INFO: Writing amalgamation header to botan_all_internal.h
INFO: Writing amalgamation source to botan_all.cpp
INFO: Botan 2.5.0 (revision git:fc3e1ed8c4f461285dc7a29dd6b69847cd61f30d) (unreleased undated) build setup is complete

FAILURE:

dev@dev-loca-xeni-x64:/tmp/botan$ make
g++ -fstack-protector -pthread   -std=c++11 -D_REENTRANT -g -momit-leaf-frame-pointer -Wall -Wextra -Wpedantic -Wstrict-aliasing -Wcast-align -Wmissing-declarations -Wpointer-arith -Wcast-qual -Wzero-as-null-pointer-constant -Wnon-virtual-dtor -Ibuild/include -c botan_all.cpp -o build/obj/lib/botan_all.o
In file included from botan_all.cpp:8:0:
botan_all.h:304:6: error: #error "Trying to compile Botan configured as x86_32 with non-x86_32 compiler."
     #error "Trying to compile Botan configured as x86_32 with non-x86_32 compiler."
      ^
Makefile:85: recipe for target 'build/obj/lib/botan_all.o' failed
make: *** [build/obj/lib/botan_all.o] Error 1

SUCCESS:

dev@dev-loca-xeni-x64:/tmp/botan$ make CXXFLAGS="-m32" LDFLAGS="-m32"
...
...
...

dev@dev-loca-xeni-x64:/tmp/botan$ ./botan-test
...
Tests complete ran 83359 tests in 130.24 sec all tests ok

[randombit]

Maybe so. Unfortunately, using -m32 on a native x86-32 compiler produces an error :( so we'd have to have special logic to detect the case of a 32-bit x86 compile on a native 64-bit host. And, simply detecting the host CPU isn't sufficient, since we could be running a 64-bit OS/kernel with a 32-bit userland, or a 64-bit userland + a 32-bit compiled gcc.

[DevSidious]

Maybe so. Unfortunately, using -m32 on a native x86-32 compiler produces an error :(

Just tested with make CXXFLAGS="-m32" LDFLAGS="-m32 on a native x86-32 compiler. No issues.

dev@dev-loca-xeni-x32:/tmp/botan$ uname -m
i686

INFO: ./configure.py invoked with options "--cpu=x86_32 --minimized-build --enable-modules=x509,ecdsa,rsa,emsa_pkcs1,sha2_64,aes --disable-shared-library --debug-mode --amalgamation --single-amalgamation-file --prefix=/opt/dev/botan/2.5.0/debug"
INFO: Guessing target OS is linux (use --os to set)
INFO: Guessing to use compiler gcc (use --cc or CXX to set)
INFO: Auto-detected compiler version 5.4
INFO: Target is gcc:5.4-linux-x86_32
INFO: Skipping (incompatible OS): darwin_secrandom getentropy win32_stats
INFO: Skipping (incompatible compiler): aes_armv8 pmull sha1_armv8 sha2_32_armv8
INFO: Skipping (not requested): adler32 aead aes_ni aes_ssse3 aont aria auto_rng bcrypt blake2 blowfish camellia cascade cast cbc cbc_mac ccm cecpq1 certstor_sql certstor_sqlite3 cfb chacha chacha20poly1305 chacha_rng chacha_sse2 checksum clmul clmul_ssse3 cmac codec_filt comb4p compression crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa dyn_load eax ecdh ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa_raw emsa_x931 fd_unix ffi filters fpe_fe1 gcm gmac gost_28147 gost_3410 gost_3411 hkdf hmac hmac_drbg hotp http_util idea idea_sse2 iso9796 kasumi kdf1 kdf1_iso18033 kdf2 keccak lion locking_allocator mce mceies md4 md5 misty1 mode_pad modes newhope nist_keywrap noekeon noekeon_simd ocb ofb par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pgp_s2k pkcs11 poly1305 poly_dbl prf_tls prf_x942 proc_walk psk_db rc4 rdrand rdrand_rng rdseed rfc3394 rfc6979 rmd160 salsa20 seed serpent serpent_simd sessions_sql sessions_sqlite3 sha1_sse2 sha1_x86 sha2_32_x86 sha3 shacal2 shacal2_simd shacal2_x86 shake shake_cipher simd siphash siv skein sm2 sm3 sm4 socket sp800_108 sp800_56a sp800_56c srp6 stateful_rng stream streebog system_rng thread_utils threefish threefish_avx2 tiger tls tls_cbc tss twofish whirlpool x919_mac xmss xtea xts
INFO: Skipping (requires external dependency): bearssl boost bzip2 lzma openssl sqlite3 tpm zlib
INFO: Loading modules: aes asn1 base base64 bigint block cpuid ec_gfp ec_group ecc_key ecdsa emsa1 emsa_pkcs1 emsa_pssr entropy hash hash_id hex kdf keypair mac mdx_hash mgf1 mp numbertheory pem pk_pad pubkey rng rsa sha1 sha2_32 sha2_64 utils x509
INFO: Using symlink to link files into build dir (use --link-method to change)
INFO: Writing amalgamation header to botan_all.h
INFO: Writing amalgamation header to botan_all_internal.h
INFO: Writing amalgamation source to botan_all.cpp
INFO: Botan 2.5.0 (revision git:fc3e1ed8c4f461285dc7a29dd6b69847cd61f30d) (unreleased undated) build setup is complete

dev@dev-loca-xeni-x32:/tmp/botan$ make CXXFLAGS="-m32" LDFLAGS="-m32" -> build succeeds

dev@dev-loca-xeni-x32:/tmp/botan$ ./botan-test
...
Tests complete ran 83359 tests in 131.12 sec all tests ok

[DevSidious]

You could factor in uname -m to get the userland bits and/or and g++ -v which is going to spit out Target: i686-linux-gnu on 32 bit userland or Target: x86_64-linux-gnu on 64 bit userland. I think these 2 variables are enough to make an informed decision.

I don't know if you find it worthwhile though.

[randombit]

Just tested with make CXXFLAGS="-m32" LDFLAGS="-m32 on a native x86-32 compiler. No issues.

Thanks for checking this. This didn't work in older GCCs but I guess they fixed it at some point. If it works in GCC 4.8 (our minimum supported version for 2.x) then we can just enable -m32 for any x86-32 build and be done with it. That is easy since it would use the same codepath we use to set -m64 for x86-64.

I'll spin up a 14.04 i386 VPS and see if it works there.

[randombit]

Ah nice it works fine with the GCC 4.8 in Ubuntu 14.04 so I think we're all set

[DevSidious]

Awesome! /me rubs hands in anticipation.

[randombit]

Fixed! Thanks again for checking on the actual behavior.

[randombit]

(BTW I should mention for completeness, Clang already uses -m32 on x86-32 builds since it never had any problem accepting -m32 flag)