search

randombit / botan

Cryptography Toolkit
https://botan.randombit.net
BSD 2-Clause "Simplified" License
2.59k stars 567 forks source link

Multiple errors when building Botan 2.4.0 for ARM with Android NDK #1451

Closed fgsalomon closed 6 years ago

fgsalomon commented 6 years ago

Hello,

We've updated Botan from version 2.0.1 to 2.4.0 and now our project it's failing to compile for ARM with the Android NDK multiple errors:

/home/user/project/botan_all.cpp: In member function 'void Botan::AES_128::armv8_encrypt_n(const uint8_t*, uint8_t*, std::size_t) const':
/home/user/project/botan_all.cpp:973:43: error: 'vaeseq_u8' was not declared in this scope
       data = vaesmcq_u8(vaeseq_u8(data, K0));
                                           ^
/home/user/project/botan_all.cpp:973:44: error: 'vaesmcq_u8' was not declared in this scope
       data = vaesmcq_u8(vaeseq_u8(data, K0));
                                            ^
/home/user/project/botan_all.cpp: At global scope:
/home/user/project/botan_all.cpp:991:81: warning: target attribute is not supported on this machine [-Wattributes]
 void AES_128::armv8_decrypt_n(const uint8_t in[], uint8_t out[], size_t blocks) const
                                                                                 ^
/home/user/project/botan_all.cpp: In member function 'void Botan::AES_128::armv8_decrypt_n(const uint8_t*, uint8_t*, std::size_t) const':
/home/user/project/botan_all.cpp:1013:44: error: 'vaesdq_u8' was not declared in this scope
       data = vaesimcq_u8(vaesdq_u8(data, K0));
                                            ^
/home/user/project/botan_all.cpp:1013:45: error: 'vaesimcq_u8' was not declared in this scope
       data = vaesimcq_u8(vaesdq_u8(data, K0));
                                             ^
/home/user/project/botan_all.cpp: At global scope:
/home/user/project/botan_all.cpp:1031:81: warning: target attribute is not supported on this machine [-Wattributes]
 void AES_192::armv8_encrypt_n(const uint8_t in[], uint8_t out[], size_t blocks) const
                                                                                 ^
/home/user/project/botan_all.cpp: In member function 'void Botan::AES_192::armv8_encrypt_n(const uint8_t*, uint8_t*, std::size_t) const':
/home/user/project/botan_all.cpp:1055:43: error: 'vaeseq_u8' was not declared in this scope
       data = vaesmcq_u8(vaeseq_u8(data, K0));
                                           ^
/home/user/project/botan_all.cpp:1055:44: error: 'vaesmcq_u8' was not declared in this scope

/home/user/project/botan_all.cpp: At global scope:
/home/user/project/botan_all.cpp:1075:81: warning: target attribute is not supported on this machine [-Wattributes]
 void AES_192::armv8_decrypt_n(const uint8_t in[], uint8_t out[], size_t blocks) const
                                                                                 ^
/home/user/project/botan_all.cpp: In member function 'void Botan::AES_192::armv8_decrypt_n(const uint8_t*, uint8_t*, std::size_t) const':
/home/user/project/botan_all.cpp:1098:44: error: 'vaesdq_u8' was not declared in this scope
       data = vaesimcq_u8(vaesdq_u8(data, K0));
                                            ^
/home/user/project/botan_all.cpp:1098:45: error: 'vaesimcq_u8' was not declared in this scope
       data = vaesimcq_u8(vaesdq_u8(data, K0));
                                             ^
/home/user/project/botan_all.cpp: At global scope:
/home/user/project/botan_all.cpp:1118:81: warning: target attribute is not supported on this machine [-Wattributes]
 void AES_256::armv8_encrypt_n(const uint8_t in[], uint8_t out[], size_t blocks) const
                                                                                 ^
/home/user/project/botan_all.cpp: In member function 'void Botan::AES_256::armv8_encrypt_n(const uint8_t*, uint8_t*, std::size_t) const':
/home/user/project/botan_all.cpp:1144:43: error: 'vaeseq_u8' was not declared in this scope
       data = vaesmcq_u8(vaeseq_u8(data, K0));
                                           ^
/home/user/project/botan_all.cpp:1144:44: error: 'vaesmcq_u8' was not declared in this scope
       data = vaesmcq_u8(vaeseq_u8(data, K0));
                                            ^
/home/user/project/botan_all.cpp: At global scope:
/home/user/project/botan_all.cpp:1166:81: warning: target attribute is not supported on this machine [-Wattributes]
 void AES_256::armv8_decrypt_n(const uint8_t in[], uint8_t out[], size_t blocks) const
                                                                                 ^
/home/user/project/botan_all.cpp: In member function 'void Botan::AES_256::armv8_decrypt_n(const uint8_t*, uint8_t*, std::size_t) const':
/home/user/project/botan_all.cpp:1192:44: error: 'vaesdq_u8' was not declared in this scope
       data = vaesimcq_u8(vaesdq_u8(data, K0));
                                            ^
/home/user/project/botan_all.cpp:1192:45: error: 'vaesimcq_u8' was not declared in this scope
       data = vaesimcq_u8(vaesdq_u8(data, K0));
                                             ^
/home/user/project/botan_all.cpp: In function 'uint64x2_t Botan::{anonymous}::gcm_multiply(uint64x2_t, uint64x2_t)':
/home/user/project/botan_all.cpp:44502:52: error: 'vmull_p64' was not declared in this scope
    uint32x4_t T0 = (uint32x4_t)vmull_p64(x_hi, H_hi);
                                                    ^
/home/user/project/botan_all.cpp: At global scope:
/home/user/project/botan_all.cpp:44516:93: warning: target attribute is not supported on this machine [-Wattributes]
                                   uint64x2_t X1, uint64x2_t X2, uint64x2_t X3, uint64x2_t X4)
                                                                                             ^
/home/user/project/botan_all.cpp: In function 'uint64x2_t Botan::{anonymous}::gcm_multiply_x4(uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t)':
/home/user/project/botan_all.cpp:44536:66: error: 'vmull_p64' was not declared in this scope
    const uint32x4_t H1_X1_lo = (uint32x4_t)vmull_p64(X1_lo, H1_lo);
                                                                  ^
/home/user/project/botan_all.cpp: At global scope:
/home/user/project/botan_all.cpp:44568:41: warning: target attribute is not supported on this machine [-Wattributes]
 inline uint8x16_t bswap_vec(uint8x16_t v)
                                         ^
/home/user/project/botan_all.cpp:44578:73: warning: target attribute is not supported on this machine [-Wattributes]
 void gcm_pmull_precompute(const uint8_t H_bytes[16], uint64_t H_pow[4*2])
                                                                         ^
/home/user/project/botan_all.cpp:44594:61: warning: target attribute is not supported on this machine [-Wattributes]
                         const uint8_t input[], size_t blocks)
                                                             ^
/home/user/project/botan_all.cpp:49526:107: warning: target attribute is not supported on this machine [-Wattributes]
 void SHA_160::sha1_armv8_compress_n(secure_vector<uint32_t>& digest, const uint8_t input8[], size_t blocks)
                                                                                                           ^
/home/user/project/botan_all.cpp: In static member function 'static void Botan::SHA_160::sha1_armv8_compress_n(Botan::secure_vector<unsigned int>&, const uint8_t*, std::size_t)':
/home/user/project/botan_all.cpp:49567:46: error: 'vsha1h_u32' was not declared in this scope
       E1 = vsha1h_u32(vgetq_lane_u32(ABCD, 0));
                                              ^
/home/user/project/botan_all.cpp:49568:40: error: 'vsha1cq_u32' was not declared in this scope
       ABCD = vsha1cq_u32(ABCD, E0, TMP0);
                                        ^
/home/user/project/botan_all.cpp:49570:44: error: 'vsha1su0q_u32' was not declared in this scope
       MSG0 = vsha1su0q_u32(MSG0, MSG1, MSG2);
                                            ^
/home/user/project/botan_all.cpp:49576:38: error: 'vsha1su1q_u32' was not declared in this scope
       MSG0 = vsha1su1q_u32(MSG0, MSG3);
                                      ^
/home/user/project/botan_all.cpp:49602:40: error: 'vsha1pq_u32' was not declared in this scope
       ABCD = vsha1pq_u32(ABCD, E1, TMP1);
                                        ^
/home/user/project/botan_all.cpp:49637:40: error: 'vsha1mq_u32' was not declared in this scope
       ABCD = vsha1mq_u32(ABCD, E0, TMP0);
                                        ^
/home/user/project/botan_all.cpp: At global scope:
/home/user/project/botan_all.cpp:49955:107: warning: target attribute is not supported on this machine [-Wattributes]
 void SHA_256::compress_digest_armv8(secure_vector<uint32_t>& digest, const uint8_t input8[], size_t blocks)
                                                                                                           ^
/home/user/project/botan_all.cpp: In static member function 'static void Botan::SHA_256::compress_digest_armv8(Botan::secure_vector<unsigned int>&, const uint8_t*, std::size_t)':
/home/user/project/botan_all.cpp:50006:40: error: 'vsha256su0q_u32' was not declared in this scope
       MSG0 = vsha256su0q_u32(MSG0, MSG1);
                                        ^
/home/user/project/botan_all.cpp:50009:50: error: 'vsha256hq_u32' was not declared in this scope
       STATE0 = vsha256hq_u32(STATE0, STATE1, TMP0);
                                                  ^
/home/user/project/botan_all.cpp:50010:49: error: 'vsha256h2q_u32' was not declared in this scope
       STATE1 = vsha256h2q_u32(STATE1, TMP2, TMP0);
                                                 ^
/home/user/project/botan_all.cpp:50011:46: error: 'vsha256su1q_u32' was not declared in this scope
       MSG0 = vsha256su1q_u32(MSG0, MSG2, MSG3);

We're using Android NDK r11c.

Botan was amalgamated like this:

 INFO: Botan 2.4.0 (revision git:1e93b85fb5a4d628b80a9812b38b842b5c8b4363) (release dated 20180108) build setup is complete
   INFO: ./configure.py invoked with options "--amalgamation --amalgamation --single-amalgamation-file --disable-modules=pkcs11 --os=android --cpu=armv8-a"
   INFO: Guessing to use compiler gcc (use --cc to set)
   INFO: Canonicalized CPU target armv8-a to arm64/armv8-a
   INFO: Auto-detected compiler version 5.4
   INFO: Target is gcc:5.4-android-arm64-armv8-a
   INFO: Skipping (dependency failure): certstor_sqlite3 rdrand sessions_sqlite3
   INFO: Skipping (disabled by user): pkcs11
   INFO: Skipping (incompatible CPU): aes_ni aes_ssse3 chacha_sse2 clmul clmul_ssse3 idea_sse2 rdrand_rng rdseed sha1_sse2 sha1_x86 sha2_32_x86 shacal2_x86 threefish_avx2
   INFO: Skipping (incompatible OS): darwin_secrandom getentropy locking_allocator win32_stats
   INFO: Skipping (no enabled compression schemes): compression
   INFO: Skipping (not requested): dyn_load
   INFO: Skipping (requires external dependency): bearssl boost bzip2 lzma openssl sqlite3 tpm zlib
   INFO: Loading modules: adler32 aead aes aes_armv8 aont aria asn1 auto_rng base base64 bcrypt bigint blake2 block blowfish camellia cascade cast cbc cbc_mac ccm cecpq1 certstor_sql cfb chacha chacha20poly1305 chacha_rng cmac codec_filt comb4p cpuid crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa eax ec_gfp ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa1 emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp http_util idea iso9796 kasumi kdf kdf1 kdf1_iso18033 kdf2 keccak keypair lion mac mce mceies md4 md5 mdx_hash mgf1 misty1 mode_pad modes mp newhope nist_keywrap noekeon noekeon_simd numbertheory ocb ofb par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pgp_s2k pk_pad pmull poly1305 poly_dbl prf_tls prf_x942 proc_walk psk_db pubkey rc4 rfc3394 rfc6979 rmd160 rng rsa salsa20 seed serpent serpent_simd sessions_sql sha1 sha1_armv8 sha2_32 sha2_32_armv8 sha2_64 sha3 shacal2 shacal2_simd shake shake_cipher simd siphash siv skein sm2 sm3 sm4 socket sp800_108 sp800_56a sp800_56c srp6 stateful_rng stream streebog system_rng threefish tiger tls tls_cbc tss twofish utils whirlpool x509 x919_mac xmss xtea xts
   INFO: Using symlink to link files into build dir (use --link-method to change)
   INFO: Writing amalgamation header to botan_all.h
   INFO: Writing amalgamation header to botan_all_internal.h
   INFO: Writing amalgamation source to botan_all.cpp
   INFO: Botan 2.4.0 (revision git:1e93b85fb5a4d628b80a9812b38b842b5c8b4363) (release dated 20180108) build setup is complete

Do you know what could be wrong? Thanks

randombit commented 6 years ago

It seems your compiler does not support the ARMv8 crypto extensions. I'm not sure why since in stock GCC they are supported from version 5.0 onwards, and it seems here we've found GCC 5.4.

One possibility is you configured the build to where CXX is the native system compiler, not the NDK compiler, and the NDK compiler is < 5.0 and does not support the extensions.

You should be able to work around this with --disable-modules=aes_armv8,sha1_armv8,sha2_32_armv8 but I'd like to better understand why our version detection failed.

fgsalomon commented 6 years ago

You are right, we were configuring the build with the native system compiler. If we set the NDK compiler using the --cc-bin flag we keep getting errors:

/home/user/botan/botan_all.cpp:44177:58: warning: target attribute is not supported on this machine [-Wattributes]
 inline uint64x2_t gcm_reduce(uint32x4_t B0, uint32x4_t B1)
                                                          ^
/home/user/botan/botan_all.cpp:44218:58: warning: target attribute is not supported on this machine [-Wattributes]
 inline uint64x2_t gcm_multiply(uint64x2_t H, uint64x2_t x)
                                                          ^
/home/user/botan/botan_all.cpp: In function 'uint64x2_t Botan::{anonymous}::gcm_multiply(uint64x2_t, uint64x2_t)':
/home/user/botan/botan_all.cpp:44227:52: error: 'vmull_p64' was not declared in this scope
    uint32x4_t T0 = (uint32x4_t)vmull_p64(x_hi, H_hi);
                                                    ^
/home/user/botan/botan_all.cpp: At global scope:
/home/user/botan/botan_all.cpp:44241:93: warning: target attribute is not supported on this machine [-Wattributes]
                                   uint64x2_t X1, uint64x2_t X2, uint64x2_t X3, uint64x2_t X4)
                                                                                             ^
/home/user/botan/botan_all.cpp: In function 'uint64x2_t Botan::{anonymous}::gcm_multiply_x4(uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t)':
/home/user/botan/botan_all.cpp:44261:66: error: 'vmull_p64' was not declared in this scope
    const uint32x4_t H1_X1_lo = (uint32x4_t)vmull_p64(X1_lo, H1_lo);
                                                                  ^
/home/user/botan/botan_all.cpp: At global scope:
/home/user/botan/botan_all.cpp:44293:41: warning: target attribute is not supported on this machine [-Wattributes]
 inline uint8x16_t bswap_vec(uint8x16_t v)
                                         ^
/home/user/botan/botan_all.cpp:44303:73: warning: target attribute is not supported on this machine [-Wattributes]
 void gcm_pmull_precompute(const uint8_t H_bytes[16], uint64_t H_pow[4*2])
                                                                         ^
/home/user/botan/botan_all.cpp:44319:61: warning: target attribute is not supported on this machine [-Wattributes]
                         const uint8_t input[], size_t blocks)
                                                             ^
/home/user/botan/botan_all.cpp:49251:107: warning: target attribute is not supported on this machine [-Wattributes]
 void SHA_160::sha1_armv8_compress_n(secure_vector<uint32_t>& digest, const uint8_t input8[], size_t blocks)
                                                                                                           ^
/home/user/botan/botan_all.cpp: In static member function 'static void Botan::SHA_160::sha1_armv8_compress_n(Botan::secure_vector<unsigned int>&, const uint8_t*, std::size_t)':
/home/user/botan/botan_all.cpp:49292:46: error: 'vsha1h_u32' was not declared in this scope
       E1 = vsha1h_u32(vgetq_lane_u32(ABCD, 0));
                                              ^
/home/user/botan/botan_all.cpp:49293:40: error: 'vsha1cq_u32' was not declared in this scope
       ABCD = vsha1cq_u32(ABCD, E0, TMP0);
                                        ^
/home/user/botan/botan_all.cpp:49295:44: error: 'vsha1su0q_u32' was not declared in this scope
       MSG0 = vsha1su0q_u32(MSG0, MSG1, MSG2);
                                            ^
/home/user/botan/botan_all.cpp:49301:38: error: 'vsha1su1q_u32' was not declared in this scope
       MSG0 = vsha1su1q_u32(MSG0, MSG3);
                                      ^
/home/user/botan/botan_all.cpp:49327:40: error: 'vsha1pq_u32' was not declared in this scope
       ABCD = vsha1pq_u32(ABCD, E1, TMP1);
                                        ^
/home/user/botan/botan_all.cpp:49362:40: error: 'vsha1mq_u32' was not declared in this scope
       ABCD = vsha1mq_u32(ABCD, E0, TMP0);
                                        ^
/home/user/botan/botan_all.cpp: At global scope:
/home/user/botan/botan_all.cpp:49680:107: warning: target attribute is not supported on this machine [-Wattributes]
 void SHA_256::compress_digest_armv8(secure_vector<uint32_t>& digest, const uint8_t input8[], size_t blocks)
                                                                                                           ^
/home/user/botan/botan_all.cpp: In static member function 'static void Botan::SHA_256::compress_digest_armv8(Botan::secure_vector<unsigned int>&, const uint8_t*, std::size_t)':
/home/user/botan/botan_all.cpp:49731:40: error: 'vsha256su0q_u32' was not declared in this scope
       MSG0 = vsha256su0q_u32(MSG0, MSG1);
                                        ^
/home/user/botan/botan_all.cpp:49734:50: error: 'vsha256hq_u32' was not declared in this scope
       STATE0 = vsha256hq_u32(STATE0, STATE1, TMP0);
                                                  ^
/home/user/botan/botan_all.cpp:49735:49: error: 'vsha256h2q_u32' was not declared in this scope
       STATE1 = vsha256h2q_u32(STATE1, TMP2, TMP0);
                                                 ^
/home/user/botan/botan_all.cpp:49736:46: error: 'vsha256su1q_u32' was not declared in this scope
       MSG0 = vsha256su1q_u32(MSG0, MSG2, MSG3);

Configuration output:

INFO: ./configure.py invoked with options "--amalgamation --single-amalgamation-file --disable-modules=pkcs11 --os=android --cpu=armv8-a --cc-bin=/android-ndk/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-g++"
   INFO: Canonicalized CPU target armv8-a to arm64/armv8-a
   INFO: Auto-detected compiler version 4.9
   INFO: Target is gcc:4.9-android-arm64-armv8-a
   INFO: Skipping (dependency failure): certstor_sqlite3 rdrand sessions_sqlite3
   INFO: Skipping (disabled by user): pkcs11
   INFO: Skipping (incompatible CPU): aes_ni aes_ssse3 chacha_sse2 clmul clmul_ssse3 idea_sse2 rdrand_rng rdseed sha1_sse2 threefish_avx2
   INFO: Skipping (incompatible OS): darwin_secrandom getentropy locking_allocator win32_stats
   INFO: Skipping (incompatible compiler): aes_armv8 sha1_x86 sha2_32_x86 shacal2_x86
   INFO: Skipping (no enabled compression schemes): compression
   INFO: Skipping (not requested): dyn_load
   INFO: Skipping (requires external dependency): bearssl boost bzip2 lzma openssl sqlite3 tpm zlib
   INFO: Loading modules: adler32 aead aes aont aria asn1 auto_rng base base64 bcrypt bigint blake2 block blowfish camellia cascade cast cbc cbc_mac ccm cecpq1 certstor_sql cfb chacha chacha20poly1305 chacha_rng cmac codec_filt comb4p cpuid crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa eax ec_gfp ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa1 emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp http_util idea iso9796 kasumi kdf kdf1 kdf1_iso18033 kdf2 keccak keypair lion mac mce mceies md4 md5 mdx_hash mgf1 misty1 mode_pad modes mp newhope nist_keywrap noekeon noekeon_simd numbertheory ocb ofb par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pgp_s2k pk_pad pmull poly1305 poly_dbl prf_tls prf_x942 proc_walk psk_db pubkey rc4 rfc3394 rfc6979 rmd160 rng rsa salsa20 seed serpent serpent_simd sessions_sql sha1 sha1_armv8 sha2_32 sha2_32_armv8 sha2_64 sha3 shacal2 shacal2_simd shake shake_cipher simd siphash siv skein sm2 sm3 sm4 socket sp800_108 sp800_56a sp800_56c srp6 stateful_rng stream streebog system_rng threefish tiger tls tls_cbc tss twofish utils whirlpool x509 x919_mac xmss xtea xts
   INFO: Using symlink to link files into build dir (use --link-method to change)
   INFO: Writing amalgamation header to botan_all.h
   INFO: Writing amalgamation header to botan_all_internal.h
   INFO: Writing amalgamation source to botan_all.cpp
   INFO: Botan 2.4.0 (revision git:1e93b85fb5a4d628b80a9812b38b842b5c8b4363) (release dated 20180108) build setup is complete

If we disable the modules with --disable-modules=aes_armv8,sha1_armv8,sha2_32_armv8 the configuration output is:

 INFO: ./configure.py invoked with options "--amalgamation --single-amalgamation-file --disable-modules=pkcs11 --os=android --cpu=armv8-a --cc-bin=/android-ndk/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-g++ --disable-modules=aes_armv8,sha1_armv8,sha2_32_armv8"
   INFO: Canonicalized CPU target armv8-a to arm64/armv8-a
   INFO: Auto-detected compiler version 4.9
   INFO: Target is gcc:4.9-android-arm64-armv8-a
   INFO: Skipping (dependency failure): certstor_sqlite3 rdrand sessions_sqlite3
   INFO: Skipping (disabled by user): aes_armv8 pkcs11 sha1_armv8 sha2_32_armv8
   INFO: Skipping (incompatible CPU): aes_ni aes_ssse3 chacha_sse2 clmul clmul_ssse3 idea_sse2 rdrand_rng rdseed sha1_sse2 threefish_avx2
   INFO: Skipping (incompatible OS): darwin_secrandom getentropy locking_allocator win32_stats
   INFO: Skipping (incompatible compiler): aes_armv8 sha1_x86 sha2_32_x86 shacal2_x86
   INFO: Skipping (no enabled compression schemes): compression
   INFO: Skipping (not requested): dyn_load
   INFO: Skipping (requires external dependency): bearssl boost bzip2 lzma openssl sqlite3 tpm zlib
   INFO: Loading modules: adler32 aead aes aont aria asn1 auto_rng base base64 bcrypt bigint blake2 block blowfish camellia cascade cast cbc cbc_mac ccm cecpq1 certstor_sql cfb chacha chacha20poly1305 chacha_rng cmac codec_filt comb4p cpuid crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa eax ec_gfp ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa1 emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp http_util idea iso9796 kasumi kdf kdf1 kdf1_iso18033 kdf2 keccak keypair lion mac mce mceies md4 md5 mdx_hash mgf1 misty1 mode_pad modes mp newhope nist_keywrap noekeon noekeon_simd numbertheory ocb ofb par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pgp_s2k pk_pad pmull poly1305 poly_dbl prf_tls prf_x942 proc_walk psk_db pubkey rc4 rfc3394 rfc6979 rmd160 rng rsa salsa20 seed serpent serpent_simd sessions_sql sha1 sha2_32 sha2_64 sha3 shacal2 shacal2_simd shake shake_cipher simd siphash siv skein sm2 sm3 sm4 socket sp800_108 sp800_56a sp800_56c srp6 stateful_rng stream streebog system_rng threefish tiger tls tls_cbc tss twofish utils whirlpool x509 x919_mac xmss xtea xts
   INFO: Using symlink to link files into build dir (use --link-method to change)
   INFO: Writing amalgamation header to botan_all.h
   INFO: Writing amalgamation header to botan_all_internal.h
   INFO: Writing amalgamation source to botan_all.cpp
   INFO: Botan 2.4.0 (revision git:1e93b85fb5a4d628b80a9812b38b842b5c8b4363) (release dated 20180108) build setup is complete

And we still get an error with vmull_p64:

/home/user/project/botan_all.cpp: In function 'uint64x2_t Botan::{anonymous}::gcm_multiply(uint64x2_t, uint64x2_t)':
/home/user/project/botan_all.cpp:44227:52: error: 'vmull_p64' was not declared in this scope
    uint32x4_t T0 = (uint32x4_t)vmull_p64(x_hi, H_hi);
                                                    ^
/home/user/project/botan_all.cpp: At global scope:
/home/user/project/botan_all.cpp:44241:93: warning: target attribute is not supported on this machine [-Wattributes]
                                   uint64x2_t X1, uint64x2_t X2, uint64x2_t X3, uint64x2_t X4)
                                                                                             ^
/home/user/project/botan_all.cpp: In function 'uint64x2_t Botan::{anonymous}::gcm_multiply_x4(uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t, uint64x2_t)':
/home/user/project/botan_all.cpp:44261:66: error: 'vmull_p64' was not declared in this scope
    const uint32x4_t H1_X1_lo = (uint32x4_t)vmull_p64(X1_lo, H1_lo);
fgsalomon commented 6 years ago

When targeting armv7 we also get an error:

/home/user/project/botan_all.cpp:11198:24: fatal error: sys/auxv.h: No such file or directory
   #include <sys/auxv.h>

Configuration output:

INFO: ./configure.py invoked with options "--amalgamation --single-amalgamation-file --disable-modules=pkcs11 --os=android --cpu=armv7 --cc-bin=/android-ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-g++ --disable-modules=aes_armv8,sha1_armv8,sha2_32_armv8"
   INFO: Canonicalized CPU target armv7 to arm32/armv7
   INFO: Auto-detected compiler version 4.9
   INFO: Target is gcc:4.9-android-arm32-armv7
   INFO: Skipping (dependency failure): certstor_sqlite3 rdrand sessions_sqlite3
   INFO: Skipping (disabled by user): aes_armv8 pkcs11 sha1_armv8 sha2_32_armv8
   INFO: Skipping (incompatible CPU): aes_ni aes_ssse3 chacha_sse2 clmul clmul_ssse3 idea_sse2 pmull rdrand_rng rdseed sha1_armv8 sha1_sse2 sha2_32_armv8 threefish_avx2
   INFO: Skipping (incompatible OS): darwin_secrandom getentropy locking_allocator win32_stats
   INFO: Skipping (incompatible compiler): aes_armv8 sha1_x86 sha2_32_x86 shacal2_x86
   INFO: Skipping (no enabled compression schemes): compression
   INFO: Skipping (not requested): dyn_load
   INFO: Skipping (requires external dependency): bearssl boost bzip2 lzma openssl sqlite3 tpm zlib
   INFO: Loading modules: adler32 aead aes aont aria asn1 auto_rng base base64 bcrypt bigint blake2 block blowfish camellia cascade cast cbc cbc_mac ccm cecpq1 certstor_sql cfb chacha chacha20poly1305 chacha_rng cmac codec_filt comb4p cpuid crc24 crc32 cryptobox ctr curve25519 des dev_random dh dl_algo dl_group dlies dsa eax ec_gfp ec_group ecc_key ecdh ecdsa ecgdsa ecies eckcdsa ed25519 elgamal eme_oaep eme_pkcs1 eme_raw emsa1 emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm gmac gost_28147 gost_3410 gost_3411 hash hash_id hex hkdf hmac hmac_drbg hotp http_util idea iso9796 kasumi kdf kdf1 kdf1_iso18033 kdf2 keccak keypair lion mac mce mceies md4 md5 mdx_hash mgf1 misty1 mode_pad modes mp newhope nist_keywrap noekeon noekeon_simd numbertheory ocb ofb par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pgp_s2k pk_pad poly1305 poly_dbl prf_tls prf_x942 proc_walk psk_db pubkey rc4 rfc3394 rfc6979 rmd160 rng rsa salsa20 seed serpent serpent_simd sessions_sql sha1 sha2_32 sha2_64 sha3 shacal2 shacal2_simd shake shake_cipher simd siphash siv skein sm2 sm3 sm4 socket sp800_108 sp800_56a sp800_56c srp6 stateful_rng stream streebog system_rng threefish tiger tls tls_cbc tss twofish utils whirlpool x509 x919_mac xmss xtea xts
   INFO: Using symlink to link files into build dir (use --link-method to change)
   INFO: Writing amalgamation header to botan_all.h
   INFO: Writing amalgamation header to botan_all_internal.h
   INFO: Writing amalgamation source to botan_all.cpp
   INFO: Botan 2.4.0 (revision git:1e93b85fb5a4d628b80a9812b38b842b5c8b4363) (release dated 20180108) build setup is complete
randombit commented 6 years ago

Interesting - GCC 4.9 is supposed to have all of these extensions already available so strange that the NDK would be missing them. I don't really know what issue could be causing this.

Re vmull error, sorry forgot about that one, can be disabled with --disable-modules=pmull. I just added a new build flag that allows disabling use of all ARMv8 crypto extensions in one go, so (in future releases) one won't have to list out every module explicitly.

Re getauxval, I believe on Android API 18 or higher is required. If that's not an option, edit src/build-data/os/android.txt and comment out getauxval line. (In master this can be done with --disable-os-feature=getauxval without having to edit anything.)

fgsalomon commented 6 years ago

Thank you @randombit, now it's working! :) As the doc you linked says those extensions have to be enabled through the -march=armv8-a+crc and -march=armv8-a+crypto options. Once enabled there aren't any compilation errors.

randombit commented 6 years ago

Oh I'm sorry I had missed this in your first output:

warning: target attribute is not supported on this machine [-Wattributes]

In the amalgamation we emit __target__ attributes so (for recent GCC and Clang) no special command line flag is necessary to enable CPU extensions like this. But I guess GCC 4.9 does not implement this attribute for AArch64.