Closed vaintroub closed 3 months ago
Hi @vaintroub
I've put up a PR to fix this: #7666
Let us know if this fixes your issues!
Thanks, Sean
I asked to test the fix in MariaDB bug report, hopefully there will be some feedback. Myself, I do not have hardware that old
Thanks @vaintroub. The problem was never noticed by us as we don't hardware 'that old' either!
So, I found a way to reproduce it, apparently just install of fresh Windows 10 to VirtualBox does it for me. AVX/AVX2, and couple of other instructions (FMA) are unsupported.
Bad news, there is still illegal instruction exception, on the same line as originally reported , line 13172 here sp_x86_64_asm.asm
IFNDEF WC_NO_CACHE_RESISTANT
_text SEGMENT READONLY PARA
sp_2048_get_from_table_32 PROC
sub rsp, 128
vmovdqu OWORD PTR [rsp], xmm6
vmovdqu OWORD PTR [rsp+16], xmm7
vmovdqu OWORD PTR [rsp+32], xmm8
Maybe, those should be movdqu
instructions, not vmovdqu
(same at the end of the function) ??
Here is the test program I'm using
#include <user_settings.h>
#include <openssl/evp.h>
#include <openssl/dh.h>
#include <openssl/bn.h>
#include <openssl/x509.h>
static EVP_PKEY *vio_keygen()
{
EVP_PKEY_CTX *ctx;
EVP_PKEY *pkey= NULL;
if (!(ctx= EVP_PKEY_CTX_new_id(EVP_PKEY_RSA, NULL)))
return NULL;
if (EVP_PKEY_keygen_init(ctx) <= 0)
goto end;
if (EVP_PKEY_CTX_set_rsa_keygen_bits(ctx,4096) <= 0)
goto end;
if (EVP_PKEY_keygen(ctx, &pkey) <= 0)
pkey= NULL; /* just in case */
end:
EVP_PKEY_CTX_free(ctx);
return pkey;
}
static X509 *vio_gencert(EVP_PKEY *pkey)
{
X509 *x;
X509_NAME *name;
if (!(x= X509_new()))
goto err;
if (!X509_set_version(x, 2))
goto err;
if (!(name= X509_get_subject_name(x)))
goto err;
if (!X509_NAME_add_entry_by_txt(name, "CN", MBSTRING_ASC, (const unsigned char *)"x",1,-1,0))
goto err;
if (!X509_set_issuer_name(x, name))
goto err;
if (!X509_gmtime_adj(X509_get_notBefore(x), 0))
goto err;
if (!X509_gmtime_adj(X509_get_notAfter(x), 60 * 60 * 24 * 365 * 10))
goto err;
if (!X509_set_pubkey(x, pkey))
goto err;
if (!X509_sign(x, pkey, EVP_sha256()))
goto err;
return x;
err:
X509_free(x);
return NULL;
}
int main()
{
printf("Hello\n");
unsigned long long start,end,freq;
QueryPerformanceFrequency((LARGE_INTEGER*)&freq);
QueryPerformanceCounter((LARGE_INTEGER*)&start);
auto key= vio_keygen();
if (!key)
{
printf("Failed to generate key\n");
return 1;
}
QueryPerformanceCounter((LARGE_INTEGER*)&end);
printf("Keygen took %f seconds\n", (end-start)/(double)freq);
auto cert= vio_gencert(key);
if (!cert)
{
printf("Failed to generate cert\n");
return 1;
}
printf("Success\n");
return 0;
}
Stacktrace just before the crash:
1 wolfssl-test.exe!sp_2048_get_from_table_32() Line 13171
2 wolfssl-test.exe!sp_2048_mod_exp_32(unsigned __int64 * r, const unsigned __int64 * a, const unsigned __int64 * e, int bits, const unsigned __int64 * m, int reduceA) Line 1572
3 wolfssl-test.exe!sp_ModExp_2048(const sp_int * base, const sp_int * exp, const sp_int * mod, sp_int * res) Line 2526
4 wolfssl-test.exe!sp_exptmod_ex(const sp_int * b, const sp_int * e, int digits, const sp_int * m, sp_int * r) Line 13753
5 wolfssl-test.exe!sp_exptmod(const sp_int * b, const sp_int * e, const sp_int * m, sp_int * r) Line 13860
6 wolfssl-test.exe!sp_prime_miller_rabin(const sp_int * a, sp_int * b, int * result, sp_int * n1, sp_int * r) Line 18792
7 wolfssl-test.exe!_sp_prime_random_trials(const sp_int * a, int trials, int * result, WC_RNG * rng) Line 19180
8 wolfssl-test.exe!sp_prime_is_prime_ex(const sp_int * a, int trials, int * result, WC_RNG * rng) Line 19263
9 wolfssl-test.exe!_CheckProbablePrime(sp_int * p, sp_int * q, sp_int * e, int nlen, int * isPrime, WC_RNG * rng) Line 4568
10 wolfssl-test.exe!wc_MakeRsaKey(RsaKey * key, int size, long e, WC_RNG * rng) Line 4868
11 wolfssl-test.exe!wolfssl_rsa_generate_key_native(WOLFSSL_RSA * rsa, int bits, WOLFSSL_BIGNUM * e, void * cb) Line 3146
12 wolfssl-test.exe!wolfSSL_RSA_generate_key(int bits, unsigned long e, void(*)(int, int, void *) cb, void * data) Line 3239
13 wolfssl-test.exe!wolfSSL_EVP_PKEY_keygen(WOLFSSL_EVP_PKEY_CTX * ctx, WOLFSSL_EVP_PKEY * * ppkey) Line 3363
14 wolfssl-test.exe!vio_keygen() Line 22
15 wolfssl-test.exe!main() Line 68
Hi @vaintroub,
I've put up another pull request that should fix this issue now. There were other functions that I targeted for the same fix and for some reason assumed get_from_table was only called with AVX1.
Let us know if this does fix the issue.
Thanks, Sean
Perfect, this patch seems to have fixed the issue. thank you!
Great to hear! Thanks for testing!
Sean
Contact Details
wlad@mariadb.com
Version
5.7.0
Description
This happens in MariaDB, which uses WolfSSL 5.7.0 (compiles as submodule), using SP_MATH_ALL with assembly support.
Most recent version of MariaDB would call wolfSSL_EVP_PKEY_keygen to generate 4096 bit large private key, to then generate an "ephemeral" server SSL certificate, if user does not provide own.
So far it worked well for us, however there is a recent bug report MDEV-34372 about invalid instruction exception, and attached crashdump confirms that instruction comes from AVX instruction set.
wolfssl\wolfcrypt\src\sp_x86_64_asm.asm, line 13172
Mostly likely, the code did not check CPUID before using it.
this is how we build it
here is generated user_settings.h
Reproduction steps
I did not reproduce myself, so the dump and the log is only thing I have, filing on behalf of our user It would be something like that
Relevant log output