ARM: aes_vperm: PKCS #8 private key decoding failed with Invalid CBC padding

[nunojpg]

In ARM Cortex-A8 if I include module aes_vperm (included by default in non minimized build), loading of a ECDSA key will throw PKCS #8 private key decoding failed with Invalid CBC padding.

[randombit]

Does the test suite pass?

What endian is the processor?

[randombit]

Also, what compiler+version?

[nunojpg]

Standard arm: little-endian.

I am cross compiling.

INFO: Target is gcc:12.2-linux-arm32 INFO: Assuming target arm32 is little endian

All tests pass. Trying to add a failing test at the moment.

[randombit]

• How was the key that you are trying to decrypt generated? (Ie was it generated by the ARM build or something else)
• If you try to decrypt that key using an x86 build, does it decrypt or not?
• Was PBKDF2 or Scrypt used? (you can check this using botan asn1print)

[nunojpg]

If I don't include module aes_vperm in the build, then loading the key works fine! And I don't need this module. It works fine as soon as I exclude it.

Key created in x86 with PBKDF2.

[randombit]

The brute force hack might be patching the vperm code to print each and every plaintext/ciphertext as it is operated on, formatted in the style of the test vector files. Then rerun the test suite with those additional tests, and one will fail.

[nunojpg]

Something like botan-test --run-long-tests pbkdf pubkey?

So far only got one to fail due to out of memory:

scrypt.vec ran 2 tests 2 FAILED
Failure 1: Test # 4 N=1048576 Output=2101CB9B6A511AAEADDBBE09CF70F881EC568D574A2FFD4DABE5EE9820ADAA478E56FD8F4BA5D09FFA1C6D927C40F4C337304049E8A952FBCBF45C6FA77A41A4 P=1 Passphrase=pleaseletmein R=8 Salt=536F6469756D43686C6F72696465 failed with exception 'std::bad_alloc' (at /opt/fr24/backend/cpp/3rdparty/botan/src/tests/test_pbkdf.cpp:231)
Failure 2: Test # 5 N=1048576 Output=E277EA2CACB23EDAFC039D229B79DC13ECEDB601D99B182A9FEDBA1E2BFB4F58 P=1 Passphrase=Rabbit R=8 Salt=4D6F757365 failed with exception 'std::bad_alloc' (at /opt/fr24/backend/cpp/3rdparty/botan/src/tests/test_pbkdf.cpp:231

[nunojpg]

[PBKDF2(HMAC(SHA-256))]
Salt = 0001020304050607
Iterations = 10000
Passphrase = xyz
Output = DEFD2987FA26A4672F4D16D98398432AD95E896BF619F6A6B8D4ED1FAF98E8B531B39FFB66966D0E115A6CD8E70B72D0

This test passes. But as soon as I added another char to xyz or increase iterations to 2x it fails.

PBKDF2(HMAC(SHA-256)) ran 3 tests in 554.30 msec 2 FAILED
Failure 1: PBKDF2(HMAC(SHA-256)) unexpected result for derived key
Produced: 73CE7906D47592FC37E5441AE986053BDFD1BBFCBA5775A4095ECDDAAFCD97C663A3670CBD690E8CC4F84C9945B77AF8
Expected: DEFD2987FA26A4672F4D16D98398432AD95E896BF619F6A6B8D4ED1FAF98E8B531B39FFB66966D0E115A6CD8E70B72D0
XOR Diff: AD3350812E53369B18A852C36A1E4611068F32974C4E8302B18A20C500557F735210F8F7DBFF6382D5A22041A2BC0828 (at /opt/fr24/backend/cpp/3rdparty/botan/src/tests/test_pbkdf.cpp:72)
Failure 2: PBKDF2(HMAC(SHA-256)) unexpected result for pwdhash derived key
Produced: 73CE7906D47592FC37E5441AE986053BDFD1BBFCBA5775A4095ECDDAAFCD97C663A3670CBD690E8CC4F84C9945B77AF8
Expected: DEFD2987FA26A4672F4D16D98398432AD95E896BF619F6A6B8D4ED1FAF98E8B531B39FFB66966D0E115A6CD8E70B72D0
XOR Diff: AD3350812E53369B18A852C36A1E4611068F32974C4E8302B18A20C500557F735210F8F7DBFF6382D5A22041A2BC0828 (at /opt/fr24/backend/cpp/3rdparty/botan/src/tests/test_pbkdf.cpp:72)
Note 1: PBKDF2(HMAC(SHA-256)) Test # 1 PBKDF2(HMAC(SHA-256)) failed Iterations=20000 Output=DEFD2987FA26A4672F4D16D98398432AD95E896BF619F6A6B8D4ED1FAF98E8B531B39FFB66966D0E115A6CD8E70B72D0 Passphrase=xyz Salt=0001020304050607

The board has 512MB of RAM (Beaglebone). I don't understand the use of vperm, but at least the failure should be explicit that it is due of out of memory, if it must be so.

[randombit]

This does not make any sense to me. PBKDF2 takes effectively fixed quantity of memory, certainly under 4KB, and consumption does not increase if either the passphrase becomes longer or the iterations increase.

[randombit]

Something like botan-test --run-long-tests pbkdf pubkey?

No. Try applying this patch:

diff --git a/src/lib/block/aes/aes_vperm/aes_vperm.cpp b/src/lib/block/aes/aes_vperm/aes_vperm.cpp
index fc70d5b85..ec6bdf92a 100644
--- a/src/lib/block/aes/aes_vperm/aes_vperm.cpp
+++ b/src/lib/block/aes/aes_vperm/aes_vperm.cpp
@@ -19,6 +19,9 @@
   #include <tmmintrin.h>
 #endif

+#include <botan/hex.h>
+#include <stdio.h>
+
 namespace Botan {

 namespace {
@@ -335,7 +338,11 @@ void AES_128::vperm_encrypt_n(const uint8_t in[], uint8_t out[], size_t blocks)
       SIMD_4x32(&m_EK[4* 9]), SIMD_4x32(&m_EK[4*10]),
    };

-   return vperm_encrypt_blocks(in, out, blocks, K, 10);
+   std::string in_hex = hex_encode(in, 16*blocks);
+   vperm_encrypt_blocks(in, out, blocks, K, 10);
+   std::string out_hex = hex_encode(out, 16*blocks);
+
+   printf("In = %s\nOut = %s\n\n", in_hex.c_str(), out_hex.c_str());
    }

 void AES_128::vperm_decrypt_n(const uint8_t in[], uint8_t out[], size_t blocks) const
@@ -347,7 +354,9 @@ void AES_128::vperm_decrypt_n(const uint8_t in[], uint8_t out[], size_t blocks)
       SIMD_4x32(&m_DK[4* 9]), SIMD_4x32(&m_DK[4*10]),
    };

-   return vperm_decrypt_blocks(in, out, blocks, K, 10);
+   std::string in_hex = hex_encode(in, 16*blocks);
+   vperm_decrypt_blocks(in, out, blocks, K, 10);
+   std::string out_hex = hex_encode(out, 16*blocks);
    }
 void AES_192::vperm_encrypt_n(const uint8_t in[], uint8_t out[], size_t blocks) const
@@ -360,7 +369,11 @@ void AES_192::vperm_encrypt_n(const uint8_t in[], uint8_t out[], size_t blocks)
       SIMD_4x32(&m_EK[4*12]),
    };

-   return vperm_encrypt_blocks(in, out, blocks, K, 12);
+   std::string in_hex = hex_encode(in, 16*blocks);
+   vperm_encrypt_blocks(in, out, blocks, K, 12);
+   std::string out_hex = hex_encode(out, 16*blocks);
+
+   printf("In = %s\nOut = %s\n\n", in_hex.c_str(), out_hex.c_str());
    }

 void AES_192::vperm_decrypt_n(const uint8_t in[], uint8_t out[], size_t blocks) const
@@ -373,7 +386,11 @@ void AES_192::vperm_decrypt_n(const uint8_t in[], uint8_t out[], size_t blocks)
       SIMD_4x32(&m_DK[4*12]),
    };

-   return vperm_decrypt_blocks(in, out, blocks, K, 12);
+   std::string in_hex = hex_encode(in, 16*blocks);
+   vperm_decrypt_blocks(in, out, blocks, K, 12);
+   std::string out_hex = hex_encode(out, 16*blocks);
+
+   printf("In = %s\nOut = %s\n\n", in_hex.c_str(), out_hex.c_str());
    }

 void AES_256::vperm_encrypt_n(const uint8_t in[], uint8_t out[], size_t blocks) const
@@ -386,7 +403,11 @@ void AES_256::vperm_encrypt_n(const uint8_t in[], uint8_t out[], size_t blocks)
       SIMD_4x32(&m_EK[4*12]), SIMD_4x32(&m_EK[4*13]), SIMD_4x32(&m_EK[4*14]),
    };

-   return vperm_encrypt_blocks(in, out, blocks, K, 14);
+   std::string in_hex = hex_encode(in, 16*blocks);
+   vperm_encrypt_blocks(in, out, blocks, K, 14);
+   std::string out_hex = hex_encode(out, 16*blocks);
+
+   printf("In = %s\nOut = %s\n\n", in_hex.c_str(), out_hex.c_str());
    }

 void AES_256::vperm_decrypt_n(const uint8_t in[], uint8_t out[], size_t blocks) const
@@ -399,7 +420,11 @@ void AES_256::vperm_decrypt_n(const uint8_t in[], uint8_t out[], size_t blocks)
       SIMD_4x32(&m_DK[4*12]), SIMD_4x32(&m_DK[4*13]), SIMD_4x32(&m_DK[4*14]),
    };

-   return vperm_decrypt_blocks(in, out, blocks, K, 14);
+   std::string in_hex = hex_encode(in, 16*blocks);
+   vperm_decrypt_blocks(in, out, blocks, K, 14);
+   std::string out_hex = hex_encode(out, 16*blocks);
+
+   printf("In = %s\nOut = %s\n\n", in_hex.c_str(), out_hex.c_str());
    }

 namespace {
@@ -514,6 +539,7 @@ SIMD_4x32 BOTAN_FUNC_ISA(BOTAN_VPERM_ISA) aes_schedule_192_smear(SIMD_4x32 x, SI

 void AES_128::vperm_key_schedule(const uint8_t keyb[], size_t /*unused*/)
    {
+   printf("[AES-128]\n\nKey = %s\n", hex_encode(keyb, 16).c_str());
    m_EK.resize(11*4);
    m_DK.resize(11*4);

@@ -540,6 +566,8 @@ void AES_128::vperm_key_schedule(const uint8_t keyb[], size_t /*unused*/)

 void AES_192::vperm_key_schedule(const uint8_t keyb[], size_t /*unused*/)
    {
+   printf("[AES-192]\n\nKey = %s\n", hex_encode(keyb, 24).c_str());
+
    m_EK.resize(13*4);
    m_DK.resize(13*4);

@@ -587,6 +615,8 @@ void AES_192::vperm_key_schedule(const uint8_t keyb[], size_t /*unused*/)

 void AES_256::vperm_key_schedule(const uint8_t keyb[], size_t /*unused*/)
    {
+   printf("[AES-256]\n\nKey = %s\n", hex_encode(keyb, 32).c_str());
+
    m_EK.resize(15*4);
    m_DK.resize(15*4);

Then run your program that is decrypting the key (or use the cli). That will print something like

[AES-256]

Key = DE79CE2CEF71071D8336CA8FC79DEA533589BB83948BCAFE7F7CBC55A83A907B
In = C4C83B5B25F6B5F47EB7FFB1A56DA8A3
Out = F9E85565411573C6F2CE4EA08EE13CD3

In = F8EE5D4FC75DBDFBF1CF49A4E3D157D1
Out = 9F7FCA4FC556AA47347F84B8B9D6C01D

In = 9E7ECE6F49D059B697BFE384DA654F1A
Out = 89B8C4457F5F8F14C6EEDFAF10C160DD

which is (almost) the correct output for the test vectors [you need to copy the Key field again for each In/Out pair] If something is wrong with the vperm code, then presumably one of these test vectors is in fact incorrect. So copying that output into src/tests/data/block/aes.vec and then re-running the block tests should indicate one test failure, which will pinpoint the issue.

[randombit]

But as soon as I added another char to xyz or increase iterations to 2x it fails.

I think you are misunderstanding how these tests work. If you increase the iterations 2x, then natually the output is not the same. The failing test you posted has the produced output, which is in fact the correct output for the inputs of that tests with 2x iterations.

[nunojpg]

Loading keys...[AES-256]

Key = 76BAF229C45EF344A01771279817B4F1DF7EFED49E28224B3E74AB046F24C0C4 In = 11A0372E185099408D4A5D31B6C90B9908E21716A100D1554DB1CF8658A8E09020B10FAE6A2028D0C6BC889F36E04FB694CE593058B22D3B2648B0026ACF100EE3A701E1FCD2F0ED83F7BCB75884DE8DDEC323F5D7BC56FE87EE8014D97C25E466A5233093B66B7A774070928BB26BEBF6E4F22443822E49C247EE92AD37E8CD Out = 7676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676767676

In = FDC7F43B6E6687556E05C536DB7E424B Out = 76767676767676767676767676767676

[randombit]

Yikes, it outputs just a constant string???

But the test suite otherwise passes :imp:

Can you try using valgrind or ASan? I'm really starting to wonder if this is some weird memory corruption.

Another thing to check: if you use the cli to decrypt the key, rather than your app, same result or no?

[nunojpg]

So, with the CLI works OK.

When I dynamically link botan to my app it also works ok, only fails when I statically link.

My app runs totally clean under asan, ubsan and valgrind.

I have now bumped GCC 13.1 and get the same results.