avzuquete commented 3 years ago

Hi,

I'm using certvalidator 0.11.1.

It cannot build a chain with the attached certificates for the one in me.der. However, it should work, as the chain was extracted from a Windows tools that examines certificate chains. me.zip

The traceback gives: File "/usr/local/lib/python3.8/site-packages/certvalidator/init.py", line 193, in validate_usage self._validate_path() File "/usr/local/lib/python3.8/site-packages/certvalidator/init.py", line 108, in _validate_path paths = self._context.certificate_registry.build_paths(self._certificate) File "/usr/local/lib/python3.8/site-packages/certvalidator/registry.py", line 314, in build_paths raise PathBuildingError(pretty_message( certvalidator.errors.PathBuildingError: Unable to build a validation path for the certificate "Common Name: ANDRÃ‰ VENTURA DA CRUZ MARNÃ”TO ZÃšQUETE; Serial Number: BI068540477; Given Name: ANDRÃ‰; Surname: VENTURA DA CRUZ MARNÃ”TO ZÃšQUETE; Organizational Unit: CidadÃ£o PortuguÃªs, Assinatura Qualificada do CidadÃ£o; Organization: CartÃ£o de CidadÃ£o; Country: PT" - no issuer matching "Common Name: ECRaizEstado 002, Organization: Sistema de CertificaÃ§Ã£o EletrÃ³nica do Estado, Country: PT" was found

wbond commented 3 years ago

Please post the PEM encoded certs in a comment here.

My hunch is that the certificate isn’t using properly encoded strings, as the error message includes what appears to be UTF-8 represented as Latin encoding.

avzuquete commented 3 years ago

Dear Will Bond,

You have all the certs in DER format.

It's very unlikely that the certs have errors, they are part of the Portuguese identity certification chains.

And yes, it's quite natural to have UTF-8 text on them, Portuguese has several diacritics.

Regards,

wbond commented 3 years ago

You have all the certs in DER format.

The issue is, I don't have time and you want help, so I am asking you do leg work. Downloading, extracting certs, converging them so that I confirm is my hunch is true is a bunch of work.

It's very unlikely that the certs have errors, they are part of the Portuguese identity certification chains.

My experience with certs is that plenty of software puts improperly encoded data into them. This is just a hunch, based on the mojibake in the error message.

And yes, it's quite natural to have UTF-8 text on them, Portuguese has several diacritics.

UTF-8 is good. The issue here is that the error message implies the certs have UTF-8 in an ASN.1 encoding that is not designed for UTF-8.

However, this is all my hunch. Once the data is laid out, it will be easier to confirm.

avzuquete commented 3 years ago

PEM files attached. me.zip

I've checked the chain step by step with openssl and it says it is good.

wbond commented 3 years ago

Can you post the PEM inline here so we can just copy paste to https://lapo.it/asn1js/?

wbond commented 3 years ago

I've checked the chain step by step with openssl and it says it is good.

That doesn't really help in this situation, as we don't use OpenSSL's chain building, nor validation code.

avzuquete commented 3 years ago

PEM certificates inline. First certificate is leaf, last is root, rest are chain aligned.

avzuquete commented 3 years ago

I've checked the chain step by step with openssl and it says it is good.

That doesn't really help in this situation, as we don't use OpenSSL's chain building, nor validation code.

I though so. But my comment was not in that direction.

I knew that that chain is correctly found (and presented) by the standard Windows certificate presenter. I just checked if openssl would behave likewise; and it does. And, as such, this reduces he probability of having encoding problems on the certificates. They may exist, though, I don't know enough of ASN.1 to discuss that with you, but it is very unlikely, given the results of those two tools.

Regards,

wbond commented 3 years ago

Can you provide the code that resulted in the backtrace?

So far it looks like the encoding on the leaf is good, so it is like that the encoding corruption is from wherever the exception was printed.

avzuquete commented 3 years ago

Can you provide the code that resulted in the backtrace?

So far it looks like the encoding on the leaf is good, so it is like that the encoding corruption is from wherever the exception was printed.

Sure. First parameter is certificate to check, second is year, them month, them day, then the rest of the certificates in the chain.

!/usr/bin/python3

import os import sys from certvalidator import CertificateValidator, ValidationContext, errors from datetime import datetime from asn1crypto.util import timezone

import sys

def main(): if len(sys.argv) < 6: print( "Usage: %s certificate year month day chain_certificate [chain_certificate]\n" % sys.argv[0], file=sys.stderr ) sys.exit( 1 )

# vc = ValidationContext( moment=datetime.now(timezone.utc) )
d = datetime(year=int(sys.argv[2]), month=int(sys.argv[3]), day=int(sys.argv[4]), tzinfo=timezone.utc )
vc = ValidationContext( moment=d )

pki = []
for i in range(5, len(sys.argv)):
    cf = open( sys.argv[i], 'rb' )
    pki.append( cf.read() )
    cf.close()

cf = open( sys.argv[1], 'rb' )
validator = CertificateValidator( cf.read(), pki, validation_context=vc )
cf.close()

for usage in ['digital_signature','non_repudiation','key_encypherment','key_agreement','key_cert_sign','crl_sign','encipher_only','decipher_only']:
    try:
        validator.validate_usage(set([usage]))
        print( "YES: %s" % (usage) )
    except (errors.PathValidationError):
        print( "NO: %s" % (usage) )
        # Print something

if name == "main": main()

3lixy commented 3 years ago

The current chain that is posted in the message https://github.com/wbond/certvalidator/issues/32#issuecomment-730719529 looks to be:

Subject: <Name(C=PT,O=Cartão de Cidadão,OU=Assinatura Qualificada do Cidadão,OU=Cidadão Português,2.5.4.4=DOS SANTOS RODRIGUES,2.5.4.42=JOÃO PEDRO,2.5.4.5=BI151540446,CN=JOÃO PEDRO DOS SANTOS RODRIGUES)> Issuer: <Name(C=PT,O=Instituto dos Registos e do Notariado I.P.,OU=Cartão de Cidadão,OU=subECEstado,CN=EC de Assinatura Digital Qualificada do Cartão de Cidadão 0015)> AKI BA:C5:DB:F9:EE:7D:7E:0D:07:EF:93:02:1D:75:40:F6:77:B4:B6:D3 SKI A6:D0:EF:D4:80:0C:05:AB:A6:3E:52:36:F4:89:18:C7:CB:C7:07:86

Subject: <Name(C=PT,O=Instituto dos Registos e do Notariado I.P.,OU=Cartão de Cidadão,OU=subECEstado,CN=EC de Assinatura Digital Qualificada do Cartão de Cidadão 0015)> Issuer: <Name(C=PT,O=SCEE - Sistema de Certificação Electrónica do Estado,OU=ECEstado,CN=Cartão de Cidadão 004)> AKI A6:D0:EF:D4:80:0C:05:AB:A6:3E:52:36:F4:89:18:C7:CB:C7:07:86 SKI 19:76:0C:AE:FE:01:61:69:44:B6:41:EB:CF:A4:38:73:C2:FB:9D:3A

Subject: <Name(C=PT,O=SCEE - Sistema de Certificação Electrónica do Estado,OU=ECEstado,CN=Cartão de Cidadão 004)> Issuer: <Name(C=PT,O=SCEE,CN=ECRaizEstado)> AKI 19:76:0C:AE:FE:01:61:69:44:B6:41:EB:CF:A4:38:73:C2:FB:9D:3A SKI 71:7F:35:DE:F5:77:71:6D:1D:12:9C:E1:90:A4:BA:F0:A9:83:8F:80

Subject: <Name(C=PT,O=SCEE,CN=ECRaizEstado)> Issuer: <Name(C=IE,O=Baltimore,OU=CyberTrust,CN=Baltimore CyberTrust Root)> AKI 71:7F:35:DE:F5:77:71:6D:1D:12:9C:E1:90:A4:BA:F0:A9:83:8F:80 SKI E5:9D:59:30:82:47:58:CC:AC:FA:08:54:36:86:7B:3A:B5:04:4D:F0

Subject: <Name(C=PT,O=MULTICERT - Serviços de Certificação Electrónica S.A.,OU=Entidade de Certificação Credenciada,CN=MULTICERT - Entidade de Certificação 001)> Issuer: <Name(C=PT,O=MULTICERT - Serviços de Certificação Electrónica S.A.,CN=MULTICERT Root Certification Authority 01)> AKI 7F:33:72:7F:4C:DA:34:C8:0E:A7:75:CB:2E:83:98:1B:06:B8:A6:90 SKI D5:39:1C:9C:5B:6F:04:AA:A2:95:4C:EF:20:DD:29:74:A4:C5:45:71

As you can see the root cert provided is not suiteable for the chain.

This is the chain that works for me but with Root Cert (https://crt.sh/?id=76) instead of the Root Cert provided.

Subject: <Name(C=PT,O=Cartão de Cidadão,OU=Assinatura Qualificada do Cidadão,OU=Cidadão Português,2.5.4.4=DOS SANTOS RODRIGUES,2.5.4.42=JOÃO PEDRO,2.5.4.5=BI151540446,CN=JOÃO PEDRO DOS SANTOS RODRIGUES)> Issuer: <Name(C=PT,O=Instituto dos Registos e do Notariado I.P.,OU=Cartão de Cidadão,OU=subECEstado,CN=EC de Assinatura Digital Qualificada do Cartão de Cidadão 0015)> AKI BA:C5:DB:F9:EE:7D:7E:0D:07:EF:93:02:1D:75:40:F6:77:B4:B6:D3 SKI A6:D0:EF:D4:80:0C:05:AB:A6:3E:52:36:F4:89:18:C7:CB:C7:07:86

Subject: <Name(C=PT,O=Instituto dos Registos e do Notariado I.P.,OU=Cartão de Cidadão,OU=subECEstado,CN=EC de Assinatura Digital Qualificada do Cartão de Cidadão 0015)> Issuer: <Name(C=PT,O=SCEE - Sistema de Certificação Electrónica do Estado,OU=ECEstado,CN=Cartão de Cidadão 004)> AKI A6:D0:EF:D4:80:0C:05:AB:A6:3E:52:36:F4:89:18:C7:CB:C7:07:86 SKI 19:76:0C:AE:FE:01:61:69:44:B6:41:EB:CF:A4:38:73:C2:FB:9D:3A

Subject: <Name(C=PT,O=SCEE - Sistema de Certificação Electrónica do Estado,OU=ECEstado,CN=Cartão de Cidadão 004)> Issuer: <Name(C=PT,O=SCEE,CN=ECRaizEstado)> AKI 19:76:0C:AE:FE:01:61:69:44:B6:41:EB:CF:A4:38:73:C2:FB:9D:3A SKI 71:7F:35:DE:F5:77:71:6D:1D:12:9C:E1:90:A4:BA:F0:A9:83:8F:80

Subject: <Name(C=PT,O=SCEE,CN=ECRaizEstado)> Issuer: <Name(C=IE,O=Baltimore,OU=CyberTrust,CN=Baltimore CyberTrust Root)> AKI 71:7F:35:DE:F5:77:71:6D:1D:12:9C:E1:90:A4:BA:F0:A9:83:8F:80 SKI E5:9D:59:30:82:47:58:CC:AC:FA:08:54:36:86:7B:3A:B5:04:4D:F0

Subject: <Name(C=IE,O=Baltimore,OU=CyberTrust,CN=Baltimore CyberTrust Root)> Issuer: <Name(C=IE,O=Baltimore,OU=CyberTrust,CN=Baltimore CyberTrust Root)> AKI E5:9D:59:30:82:47:58:CC:AC:FA:08:54:36:86:7B:3A:B5:04:4D:F0 SKI

However when using this correct ROOT cert you will encounter the issue fixed in https://github.com/wbond/certvalidator/pull/28.

chrisdlangton commented 2 years ago

I'm also having trouble validating the cert chain and am happy to open a separate issue if needed but i'll add to this one until asked to make a separate specific issue for my scenario.

To make things easy, i'll test against badssl.com so we can avoid zipping certs or posting walls of text. choose any host you like for this problem, they all have a chain.

the chain obtained using:

def get_peer_certificate_chain(domain_name):
    peer_certificate_chain = []
    for method in [SSL.TLSv1_2_METHOD, SSL.TLSv1_1_METHOD, SSL.TLSv1_METHOD, SSL.SSLv23_METHOD]:
        context = SSL.Context(method=method)
        for bundle in [requests.certs.where()]:
            context.load_verify_locations(cafile=bundle)
        sock = SSL.Connection(context=context, socket=socket(AF_INET, SOCK_STREAM))
        sock.settimeout(5)
        sock.set_tlsext_host_name(domain_name.encode())
        try:
            sock.connect((domain_name, 443))
            sock.setblocking(1)
            sock.do_handshake()
            for (_, cert) in enumerate(sock.get_peer_cert_chain()):
                peer_certificate_chain.append(cert)
            sock.shutdown()
            sock.close()
            break
        except Exception as ex:
            logger.exception(ex)
            sock.shutdown()
            sock.close()
    return peer_certificate_chain

but CertificateValidator takes 3 other types for intermediate_certs - and I chose pem encoding:

intermediate_certs = []
for cert in peer_certificate_chain(host):
    intermediate_certs.append(dump_certificate(FILETYPE_PEM, cert))

okay on with the example;

ctx = ValidationContext(allow_fetching=True, revocation_mode='hard-fail', weak_hash_algos=set(["md2", "md5", "sha1"]))
der = sock.getpeercert(True) # should be self explanatory how to create a socket using get_peer_certificate_chain example
x509 = self.server_certificate.to_cryptography() # I use the cryptography lib extensively
# later, I have access to the cryptography lib but CertificateValidator requires the der
der = x509.tbs_certificate_bytes
validator = CertificateValidator(der, validation_context=ctx, intermediate_certs=intermediate_certs)
validator.validate_usage(
    key_usage=set(['digital_signature', 'crl_sign']),
    extended_key_usage=set(['ocsp_signing']),
)

this is the error I get: Error parsing asn1crypto.algos.SignedDigestAlgorithm - method should have been constructed, but primitive was found\n while parsing asn1crypto.x509.Certificate

The trace:

  File "/srv/app/.local/lib/python3.8/site-packages/certvalidator/__init__.py", line 193, in validate_usage
    self._validate_path()
  File "/srv/app/.local/lib/python3.8/site-packages/certvalidator/__init__.py", line 98, in _validate_path
    if self._certificate.hash_algo in self._context.weak_hash_algos:
  File "/srv/app/.local/lib/python3.8/site-packages/asn1crypto/x509.py", line 2524, in hash_algo
    return self['signature_algorithm'].hash_algo
  File "/srv/app/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 3536, in __getitem__
    raise e
  File "/srv/app/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 3531, in __getitem__
    return self._lazy_child(key)
  File "/srv/app/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 3478, in _lazy_child
    child = self.children[index] = _build(*child)
  File "/srv/app/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 5551, in _build

I have refactored to remove the usage of cryptography lib entirely, i.e.

# create sock using get_peer_certificate_chain example
host = 'mozilla-modern.badssl.com'
intermediate_certs = []
for cert in get_peer_certificate_chain(host):
    intermediate_certs.append(dump_certificate(FILETYPE_PEM, cert))
ctx = ValidationContext(allow_fetching=True, revocation_mode='hard-fail', weak_hash_algos=set(["md2", "md5", "sha1"]))
der = sock.getpeercert(True)
validator = CertificateValidator(der, validation_context=ctx, intermediate_certs=intermediate_certs)
validator.validate_usage(
    key_usage=set(['digital_signature', 'crl_sign']),
    extended_key_usage=set(['ocsp_signing']),
)

But no change, the same exception occurs. I've spent over 12 hours on this, the scenario is pretty complete now, am at a loss what to try next so I'm keen for any advice at all.

wbond commented 2 years ago

@stof What version of asn1crypto are you using? The error message indicates that asn1crypto is finding an ASN.1 construction it doesn't expect in one of the certificates.

stof commented 2 years ago

@wbond you mentioned the wrong person

wbond commented 2 years ago

Sorry about that!

wbond commented 2 years ago

@chrisdlangton See above ^

wbond / certvalidator

Chain building in certificate validation #32

!/usr/bin/python3