SSL BIO is broken for internationalized domains

[asvetlov]

BPO	31872
Nosy	@pitrou, @tiran, @njsmith, @asvetlov
Superseder	bpo-28414: SSL match_hostname fails for internationalized domain names

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/tiran' closed_at = created_at = labels = ['expert-SSL', 'type-bug', '3.7'] title = 'SSL BIO is broken for internationalized domains' updated_at = user = 'https://github.com/asvetlov' ``` bugs.python.org fields: ```python activity = actor = 'asvetlov' assignee = 'christian.heimes' closed = True closed_date = closer = 'asvetlov' components = ['SSL'] creation = creator = 'asvetlov' dependencies = [] files = [] hgrepos = [] issue_num = 31872 keywords = [] message_count = 3.0 messages = ['305042', '305404', '305407'] nosy_count = 4.0 nosy_names = ['pitrou', 'christian.heimes', 'njs', 'asvetlov'] pr_nums = [] priority = 'normal' resolution = 'duplicate' stage = 'resolved' status = 'closed' superseder = '28414' type = 'behavior' url = 'https://bugs.python.org/issue31872' versions = ['Python 3.6', 'Python 3.7'] ```

[asvetlov]

SSLContext.wrap_bio creates a new SSLObject instance with passed server_hostname. The name becomes IDNA-decoded: 'xn--2qq421aovb6v1e3pu.xn--j6w193g' is converted to '雜草工作室.香港' by SSLObject constructor.

Than on SSL handshake ssl.match_hostname() is called with sslobject.server_hostname parameter ('雜草工作室.香港' in my example).

But certificate for the site is contains IDNA-encoded DNS names:

{'OCSP': ('http://ocsp.comodoca4.com',),
 'caIssuers': ('http://crt.comodoca4.com/COMODOECCDomainValidationSecureServerCA2.crt',),
 'crlDistributionPoints': ('http://crl.comodoca4.com/COMODOECCDomainValidationSecureServerCA2.crl',),
 'issuer': ((('countryName', 'GB'),),
            (('stateOrProvinceName', 'Greater Manchester'),),
            (('localityName', 'Salford'),),
            (('organizationName', 'COMODO CA Limited'),),
            (('commonName',
              'COMODO ECC Domain Validation Secure Server CA 2'),)),
 'notAfter': 'Mar 28 23:59:59 2018 GMT',
 'notBefore': 'Sep 19 00:00:00 2017 GMT',
 'serialNumber': 'FBFE0BF7CACA6DDC15968410BAA1908D',
 'subject': ((('organizationalUnitName', 'Domain Control Validated'),),
             (('organizationalUnitName', 'PositiveSSL Multi-Domain'),),
             (('commonName', 'sni38752.cloudflaressl.com'),)),
 'subjectAltName': (('DNS', 'sni38752.cloudflaressl.com'),
                    ('DNS', '*.1km.hk'),
                    ('DNS', '*.acg-cafe.com'),
                    ('DNS', '*.acgapp.moe'),
                    ('DNS', '*.acgapp.net'),
                    ('DNS', '*.cosmatch.org'),
                    ('DNS', '*.dirimusik.com'),
                    ('DNS', '*.dirimusik.info'),
                    ('DNS', '*.downloadlagi.club'),
                    ('DNS', '*.downloadlaguaz.info'),
                    ('DNS', '*.farmprecision.com'),
                    ('DNS', '*.glowecommercialphotography.co.uk'),
                    ('DNS', '*.hypertechglobal.com'),
                    ('DNS', '*.hypertechglobal.hk'),
                    ('DNS', '*.infoku.download'),
                    ('DNS', '*.inimp3.com'),
                    ('DNS', '*.luciafitness.com.au'),
                    ('DNS', '*.merdeka.news'),
                    ('DNS', '*.promisecos.com'),
                    ('DNS', '*.promisecos.hk'),
                    ('DNS', '*.ps9architects.com'),
                    ('DNS', '*.rubaxeu.gq'),
                    ('DNS', '*.ruth-fox.com'),
                    ('DNS', '*.simmit.net.au'),
                    ('DNS', '*.startss.today'),
                    ('DNS', '*.xn--2qq421aovb6v1e3pu.xn--j6w193g'),
                    ('DNS', '*.xn--hhrw16aw6jizf.xn--j6w193g'),
                    ('DNS', '1km.hk'),
                    ('DNS', 'acg-cafe.com'),
                    ('DNS', 'acgapp.moe'),
                    ('DNS', 'acgapp.net'),
                    ('DNS', 'cosmatch.org'),
                    ('DNS', 'dirimusik.com'),
                    ('DNS', 'dirimusik.info'),
                    ('DNS', 'downloadlagi.club'),
                    ('DNS', 'downloadlaguaz.info'),
                    ('DNS', 'farmprecision.com'),
                    ('DNS', 'glowecommercialphotography.co.uk'),
                    ('DNS', 'hypertechglobal.com'),
                    ('DNS', 'hypertechglobal.hk'),
                    ('DNS', 'infoku.download'),
                    ('DNS', 'inimp3.com'),
                    ('DNS', 'luciafitness.com.au'),
                    ('DNS', 'merdeka.news'),
                    ('DNS', 'promisecos.com'),
                    ('DNS', 'promisecos.hk'),
                    ('DNS', 'ps9architects.com'),
                    ('DNS', 'rubaxeu.gq'),
                    ('DNS', 'ruth-fox.com'),
                    ('DNS', 'simmit.net.au'),
                    ('DNS', 'startss.today'),
                    ('DNS', 'xn--2qq421aovb6v1e3pu.xn--j6w193g'),
                    ('DNS', 'xn--hhrw16aw6jizf.xn--j6w193g')),
 'version': 3}

Match '雜草工作室.香港' to ('DNS', 'xn--2qq421aovb6v1e3pu.xn--j6w193g') obviously fails.

I see two possible solutions:

1. Always do IDNA encoding for server_hostname stored in ssl object.
2. Do two checks for both IDNA and original server hostname values. I don't sure if certificates always use IDNA-encoded DNS names only.

The fix is trivial, I'll make a Pull Request after choosing what option we want to support. Personally I'm inclined to second one.

P.S. requests library is not affected because it uses ssl.wrap_socket. The bug is reproducible for asyncio only (and maybe Tornado with asyncio IOLoop).

[njsmith]

I believe https://github.com/python/cpython/pull/3010 is the fix you're looking for.

[asvetlov]

Duplicate of bpo-28414

Nathaniel thanks for Pull Request!