Map certs with ITUT X509 to our RSA implementation

Description of changes:

There exists a rarer OID of RSA public keys which comes from the ITU-T version of X.509 (links below). Links:

OpenSSL accepts the sign_nid and pkey_nid for this draft.

NID_md5WithRSA
NID_sha1WithRSA
NID_rsa

We can parse both sign_nids correctly. But we lack the logic to interpret NID_rsa in a certificate or map the parsed sign_nids to any RSA implementation. OpenSSL maps the corresponding nids from the same draft, which means NID_md5WithRSA and NID_sha1WithRSA are mapped to NID_rsa and an additional EVP_PKEY_RSA2 is used to represent NID_rsa. The additional EVP_PKEY type introduces additional complexities however and the underlying functionality in EVP_PKEY_RSA2 is identical to it's more common EVP_PKEY_RSA counterpart.

We've been burned by multiple OpenSSL cert parsing differences in the past, so we should have logic to understand certs that have NID_rsa. We don't want to duplicate all the function pointers for EVP_PKEY_RSA2 however. Since the underlying RSA functions pointers are the same, I think it should be OK for us to

Map NID_md5WithRSA and NID_sha1WithRSA to our RSA implementation (NID_rsa_encryption).
Map NID_rsa to EVP_PKEY_RSA when parsing certificates.

Unfortunately, there's some stricter parsing in rsa_pub_decode that I've had to remove since it doesn't apply to both RFCs for RSA public keys. This is aligning with OpenSSL, which happens to ignore both of the early parameters specified in either.

Documented in https://github.com/aws/aws-lc/commit/f6094e05efd294e15fe7f2e430f391445ee546bb
OpenSSL code (which ignores the parameter): https://github.com/openssl/openssl/blob/8e0d479b98357bb20ab1bd073cf75f7d42531553/crypto/rsa/rsa_ameth.c#L76-L90

Call-outs:

Although we have EVP_PKEY_RSA2 and we could map the logic for that to EVP_PKEY_RSA in EVP_PKEY_type, I've intentionally left that out for now. We still want to discourage people from using this form, we just need to understand how to parse certificates that have it.

Relevant historic commits:

Removed support for NID_rsa: https://github.com/aws/aws-lc/commit/f6094e05efd294e15fe7f2e430f391445ee546bb
Sigalgs mapping removed: https://github.com/aws/aws-lc/commit/720ff53d0785b37d468fc14178d419023798ccfd
RSA stricter parsing introduced: https://github.com/aws/aws-lc/commit/68772b31b07827793827e45ea81e8035269774c1

Testing:

WIP

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

aws / aws-lc