search

randombit / botan

Cryptography Toolkit
https://botan.randombit.net
BSD 2-Clause "Simplified" License
2.58k stars 566 forks source link

X.509 self signed certificate not recognized #1042

Closed reneme closed 7 years ago

reneme commented 7 years ago

We have a root CA certificate that should be self-signed (according to OpenSSL [1]). However, Botan neither recognizes it as self-signed nor manages to verify its signature. Random other certificates work perfectly well, though.

What could be the problem here?

Botan::DataSource_Memory ds(
    "-----BEGIN CERTIFICATE-----"
    "MIIEJDCCAwygAwIBAgIDD+RPMA0GCSqGSIb3DQEBCwUAMEoxCzAJBgNVBAYTAkRF"
    "MRUwEwYDVQQKEwxELVRydXN0IEdtYkgxJDAiBgNVBAMTG0QtVFJVU1QgUm9vdCBU"
    "ZXN0IENBIDMgMjAxMzAeFw0xMzA5MjAwODI1NTFaFw0yODA5MjAwODI1NTFaMEox"
    "CzAJBgNVBAYTAkRFMRUwEwYDVQQKEwxELVRydXN0IEdtYkgxJDAiBgNVBAMTG0Qt"
    "VFJVU1QgUm9vdCBUZXN0IENBIDMgMjAxMzCCASIwDQYJKoZIhvcNAQEBBQADggEP"
    "ADCCAQoCggEBAKgFHZ8q2nd9E2RmgMURUtwcNJt4t5ZRzYvg/ABnORaJRnef0Jen"
    "C8QqXoKYoji/x1Udh93EndVvy4BAJ6GZny7ij1J1Cb9RKniTr5Xheggd4L4Dp+Nb"
    "IZvodk8Co0WSsaj69IK/RQYjreseaxyN5pJJvIP3A1buDZO3DXd/936WG1sRxDmh"
    "+OgKFmsd7TOuimR2rPym1+j+KhlBMmLKNK5rWHZWF/zQXYVjjc6uWrx+mwE2xoTH"
    "bXAqi5VC4f0Mh/e/hH2+8F7+l62Cbsq+45eT8Nmz29DY6akLlwGPx7mue6ydOWyI"
    "ljpl+bBKKBJD6dMM8eR2x+AtjQWhKBQcg3sCAwEAAaOCAREwggENMA8GA1UdEwEB"
    "/wQFMAMBAf8wHQYDVR0OBBYEFBzLxNltYSUqE+RdRhwZOf5Ybj6nMA4GA1UdDwEB"
    "/wQEAwIBBjCBygYDVR0fBIHCMIG/MHugeaB3hnVsZGFwOi8vZGlyZWN0b3J5LmQt"
    "dHJ1c3QubmV0L0NOPUQtVFJVU1QlMjBSb290JTIwVGVzdCUyMENBJTIwMyUyMDIw"
    "MTMsTz1ELVRydXN0JTIwR21iSCxDPURFP2NlcnRpZmljYXRlcmV2b2NhdGlvbmxp"
    "c3QwQKA+oDyGOmh0dHA6Ly9jcmwuZC10cnVzdC5uZXQvY3JsL2QtdHJ1c3Rfcm9v"
    "dF90ZXN0X2NhXzNfMjAxMy5jcmwwDQYJKoZIhvcNAQELBQADggEBAJiB8VPPBXPD"
    "zk4Rq9hV/gtangTut9sjUFvnd0RLRLH3MxbMy4g6HmwfUb+1v607uP5dKW+tyhtl"
    "iTQofFtJOYjmPWZztYY5Z3GORNYGBu9cf2/21IJ70XXy5ILcgJBiiQvdosiwZ0ly"
    "68T5Svcuu2A02EtdSYN5kvjrnoMR/gzXqIStZ5FMLerdIR8OAKd9sAWR9MSUU9QL"
    "dtENGQWWrJSz2cNze4D8eV/jQsbrDf2krps8bWUOs4hr6hvIHwMherBSUGl4ejKa"
    "4pX/LGrD44gL/Rf6acgbUvp+jsX8tXHAZBCZ6SeCTH/W1TJ8y5KX+C93v72EM3AD"
    "sOREwZ0wWok="
    "-----END CERTIFICATE-----");

Botan::X509_Certificate cert(ds);
assert(cert.check_signature(cert.subject_public_key()));  // fails
assert(cert.is_self_signed());                            // fails

Thanks in advance.

[1] openssl verify -CAfile root.pem -CApath root.pem -verbose root.pem -> root.pem: OK

xqp commented 7 years ago

I have tested the certificate programmatically with openssl

    X509 *rootCert = "";  // use the certificate from Reneme
    EVP_PKEY *pubkey = X509_get_pubkey(rootCert);
    int retValue = X509_verify(rootCert, pubkey);
    EVP_PKEY_free(pubkey);
    X509_free(rootCert);
    assert(retValue == 1); //successfull
randombit commented 7 years ago

When verifying the cert, RSA message recovery works but PKCS padding verification fails to match the expected result so the signature is rejected. That the certificate is not considered self-signed is a result of the signature check failing (see #634 for background on this).

The expected value during RSA message recovery is

01FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF003031300D06096086480165030402010500042054F70EACBE78806951298CD3BC31CA9C8AD374D107D0E17DD82DC9BDB9C0B330 vs the encoded 01FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00302F300B0609608648016503040201042054F70EACBE78806951298CD3BC31CA9C8AD374D107D0E17DD82DC9BDB9C0B330

The last 32 bytes (the SHA-256 hash itself) match. The problem is in the certificates encoding of the hash ID. In PKCSv1.5 signatures, the hash used is encoded as the DER encoding of an ASN.1 AlgorithmIdentifier and as such is a fixed byte sequence (see https://tools.ietf.org/html/rfc8017#section-9.2 "the DER encoding T of the DigestInfo value is equal to the following") encoding the hash's object identifier plus "NULL" parameters. But this certificate instead omits the parameters field entirely, as can be seen with:

   {
   Botan::BER_Decoder dec(Botan::hex_decode("300D060960864801650304020105000420"));
   Botan::AlgorithmIdentifier id;
   dec.decode(id);
   std::cout << Botan::hex_encode(id.parameters) << "\n";
   std::cout << id.oid.as_string() << "\n";
   }

   {
      Botan::BER_Decoder dec(Botan::hex_decode("300B06096086480165030402010420"));
   Botan::AlgorithmIdentifier id;
   dec.decode(id);
   std::cout << Botan::hex_encode(id.parameters) << "\n";
   std::cout << id.oid.as_string() << "\n";
   }

which outputs

0500
2.16.840.1.101.3.4.2.1

2.16.840.1.101.3.4.2.1

The 0500 above is the ASN.1 encoding of NULL parameters, vs the blank line of "empty parameters field".

This works in OpenSSL because I guess instead of recomputing the expected message representative and comparing them, it actually decodes the PKCS algorithm identifier with the ASN.1 parser. Which is interesting as I thought the broad consensus (among crypto implementers) was that doing that during PKCS signature verification was a Bad Idea. NSS seems to follow basically the recompute-and-compare logic, but has a param unsafeAllowMissingParameters which causes it to recompute-and-compare using hash identifiers both with NULL params and missing params. I don't know if NSS enables unsafeAllowMissingParameters during validation or not. Golang follows the recompute-and-compare method (https://github.com/golang/go/blob/master/src/crypto/rsa/pkcs1v15.go#L204) and so will also reject this cert's signature as invalid.

@neusdan @securitykernel Thoughts on this one? Because it is a trust root technically we don't need to validate the signature, but that conflicts with the self-issued check. We could go the NSS route of checking for hash IDs both with and without NULL params, but I'd like to better understand the security implications there.

securitykernel commented 7 years ago

Thanks for the detailed (and interesting) analysis. FWIW, GnuTLS also fails to verify the signature:

$ certtool -e --infile d-trust_test_root_ca_3_2013.cer
Loaded 1 certificates, 1 CAs and 0 CRLs

    Subject: C=DE,O=D-Trust GmbH,CN=D-TRUST Root Test CA 3 2013
    Issuer: C=DE,O=D-Trust GmbH,CN=D-TRUST Root Test CA 3 2013
    Checked against: C=DE,O=D-Trust GmbH,CN=D-TRUST Root Test CA 3 2013
    Output: Not verified. The certificate is NOT trusted. The signature in the certificate is invalid. 

Chain verification output: Not verified. The certificate is NOT trusted. The signature in the certificate is invalid.

The certificate is the D-TRUST Test Root CA 3 2013 from here. Ok, RFC 8017 seems pretty clear on this and this certificate clearly does not conform to it. It would be interesting to know how it was generated.

Regarding NSS, I found https://bugzilla.mozilla.org/show_bug.cgi?id=1296986 where they discuss removing the unsafeAllowMissingParameters option, so the consensus seems to be not to support such certificates. I would prefer to get in contact with D-TRUST and give them notice that their certificate is invalid. D-TRUST is the trust center of Bundesdruckerei - a German state-owned IT security vendor. @reneme What do you think?

As a side note, Botan CLI is not able to print the certificate contents (at least in 2.0.1) and we should probably fix that:

/botan cert_info d-trust_test_root_ca_3_2013.cer
X509_Certificate::to_string failed: Data_Store::get1: More than one value for CRL.DistributionPoint
securitykernel commented 7 years ago

Interestingly, I can successfully verify the downloaded https://www.d-trust.net/fileadmin/programme/Zertifikate/2013/d-trust_test_root_ca_3_2013.cer. It has a different serial than the one pasted by @reneme above and was generated 6 days later. Maybe D-TRUST noticed the mistake and regenerated the Test CA cert.

mouse07410 commented 7 years ago

Can you explain the potential risk of treating empty list of parameters as NULL list of parameters? In both cases nothing is present.

hrantzsch commented 7 years ago

From version 1.1.0 openSSL too fails to validate the signature. Trying to validate a certificate that was signed by this D-Trust root CA:

$ openssl version
OpenSSL 1.1.0e  16 Feb 2017
$ openssl verify -CAfile ca user.pem
C = DE, O = D-Trust GmbH, CN = D-TRUST Application Certificates Test CA 3-1 2013
error 7 at 1 depth lookup: certificate signature failure
error user.pem: verification failed
140559040778112:error:04091068:rsa routines:int_rsa_verify:bad signature:crypto/rsa/rsa_sign.c:220:
140559040778112:error:0D0C5006:asn1 encoding routines:ASN1_item_verify:EVP lib:crypto/asn1/a_verify.c:174:
$ openssl-1.0 version
OpenSSL 1.0.2k  26 Jan 2017
$ openssl-1.0 verify -CAfile ca user.pem
user.pem: OK
securitykernel commented 7 years ago

Can you explain the potential risk of treating empty list of parameters as NULL list of parameters? In both cases nothing is present.

I can not. Maybe there is a risk, maybe there isn't, we don't know. But in any case such a certificate does not conform to RFC. And the general consensus in the Internet PKI seems to be to reject such certificates. A recent example is the CN/SAN rejection in Chrome 58.

@reneme Can you try the current D-TRUST Test Root CA 3 2013 from here and report if it works for you?

reneme commented 7 years ago

@securitykernel The problem is, that this certificate has a subtly different CN:

In this thread: D-TRUST Root Test CA 3 2013
From your link: D-TRUST Test Root CA 3 2013
                        ^~~~~~~~^

And the certificate I am trying to verify with it is signed with the one posted above. I am in contact with D-Trust but didn't hear back from them yet. I'll keep you updated.

reneme commented 7 years ago

Coming back to this and after some eMail exchange with D-TRUST, they seem to be aware of the problem. The certificate is part of their testing infrastructure and apparently we were the only ones to complain about it.

Anyway, we switched our application to a production-grade CA inside D-TRUST now and those certificates work perfectly fine.

Thank you very much for your detailed analysis.

closing as solved

securitykernel commented 7 years ago

Thanks for providing the feedback.