Fix 3.0 compatability issues

microsoft / SymCrypt-OpenSSL

OpenSSL engine for use with SymCrypt cryptographic library

MIT License

43 stars 11 forks source link

Fix 3.0 compatability issues #73

Closed mamckee closed 8 months ago

mamckee commented 8 months ago

RSA_PSS_SALTLEN_AUTO_DIGEST_MAX and EVP_MD_CTX_dup weren't added until OpenSSL 3.1. This PR fixes builds for 3.0 by:

Wrapping RSA_PSS_SALTLEN_AUTO_DIGEST_MAX with #ifdef (same thing as in ScosslCommon)
Define EVP_MD_CTX_dup using the official 3.1 implementation if building on 3.0

samuel-lee-msft commented 8 months ago

Hrmm, this is a bit unfortunate if an application is compiled using OSSL_PKEY_RSA_PSS_SALT_LEN_AUTO_DIGEST_MAX but the dynamically linked OpenSSL / provider is only 3.0 so is not aware; rather than using a maximum length, it sets cbSalt to 0 on sign / verify?

Maybe a bug in upstream.

Refers to: SymCryptProvider/src/signature/p_scossl_rsa_signature.c:473 in 1d6b7dc. [](commit_id = 1d6b7dcdc92b9f22f7c891ca0d369be0c33dbbb0, deletion_comment = False)

samuel-lee-msft commented 8 months ago

                    return 0;

NP: return SCOSSL_FAILURE;

Refers to: SymCryptProvider/src/signature/p_scossl_rsa_signature.c:631 in 1d6b7dc. [](commit_id = 1d6b7dcdc92b9f22f7c891ca0d369be0c33dbbb0, deletion_comment = False)