Support TLS Encrypted ClientHello (ECH)

[69d7615d-9f6d-4a6e-a3fe-7145b096f05c]

BPO	45567
Nosy	@tiran, @eighthave

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/tiran' closed_at = None created_at = labels = ['expert-SSL', 'type-feature'] title = 'Support TLS Encrypted ClientHello (ECH)' updated_at = user = 'https://github.com/eighthave' ``` bugs.python.org fields: ```python activity = actor = 'eighthave' assignee = 'christian.heimes' closed = False closed_date = None closer = None components = ['SSL'] creation = creator = 'eighthave' dependencies = [] files = [] hgrepos = [] issue_num = 45567 keywords = [] message_count = 3.0 messages = ['404732', '404749', '405513'] nosy_count = 2.0 nosy_names = ['christian.heimes', 'eighthave'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue45567' versions = [] ```

[69d7615d-9f6d-4a6e-a3fe-7145b096f05c]

The next version of the IETF-standardized TLS protocol is known as Encrypted ClientHello (ECH) [1] formerly known as Encrypted SNI (ESNI). This ticket collects information for ECH support, and tracks which APIs have to be added to Python in order to implement ECH in Python's ssl module. ECH is built on top of TLSv1.3 and completes the unfinished work from the TLSv1.3 effort. It is now in draft-13 and there are many implementations that are interoperating. ECH is working for openssl[2], boringssl[3], nginx, Apache HTTPD, lighttpd, HAProxy, Conscrypt[4], curl, and more. There is work underway in Firefox [5] and Chromium [6]. It has been sketched out for OkHTTP [7]. Early versions of the standard, known as ESNI, have been deployed in Firefox releases and some production web services. ECH works in conjunction with the new DNS RR Types HTTPS and SVCB [8]. This means that DNS needs to be handled a bit differently.

As far as I understand it, the ssl module has to gain additional features:

1. HTTPS/SVCB DNS queries for setting up TLS connection with ECH.
2. A way to provide ECH Config Lists as bytes directly to ssl clients.
3. A callback that gets called whenever ECH negotiation fails and the server offers a "Retry Config".
4. A method to ensure encrypted DNS is used so all metadata is encrypted.

OpenSSL does not implement the necessary APIs yet. Stephen Farrell's development OpenSSL fork [9] implements ECH and has been used in Apache HTTPD, nginx, lighttpd, HAProxy, and curl implementations.

The TLS WG maintain a page with information about other implementations: https://github.com/tlswg/draft-ietf-tls-esni/wiki/Implementations

[1] https://www.ietf.org/archive/id/draft-ietf-tls-esni-13.html [2] https://github.com/openssl/openssl/issues/7482 [3] https://bugs.chromium.org/p/boringssl/issues/detail?id=275 [5] https://bugzilla.mozilla.org/show_bug.cgi?id=1725938 [6] https://bugs.chromium.org/p/chromium/issues/detail?id=1091403 [6] https://github.com/google/conscrypt/issues/730 [7] https://github.com/square/okhttp/issues/6539 [8] https://www.ietf.org/archive/id/draft-ietf-dnsop-svcb-https-07.html [9] https://github.com/sftcd/openssl

[tiran]

Thanks for filing this feature request!

The DNS lookup part is out of scope for the ssl module. I don't want to get into the DNS business. At $WORK I work on BIND, DNSSEC and DNS over TLS. Secure DNS (and DNS in general) is already complicated when you control the entire stack and only have to deal with one family of Linux distros. AFAIK there isn't even a platform-independent way to perform lookups with abitrary RRTYPEs. res_nquery() is only available on some platforms and doesn't work reliable with some libcs. I have had issues with EDNS0 on musl in the past. KRB5KDC SRV looks with large responses where unreliable. Let's offload the DNS part of consumers of the ssl module. They can use python-dns or c-ares.

The ECH part and callback look sensible, though. I'll include the APIs as soon as OpenSSL support them.

[69d7615d-9f6d-4a6e-a3fe-7145b096f05c]

I agree with all you say, but I think it is important to not rule out handling HTTPS/SVCB DNS here. It can happen at a later stage though. What you propose works great for the first step.

If handling the DNS is punted to some external library, that means that each client app will have to implement this itself, and its non-trivial. In that scenario, very few clients will implement it. This makes me believe that this should be implemented in the core. Granted, its an open question whether the ssl module is the right place. For example, it could makes more sense to handle HTTPS/SVCB DNS in the HTTP libs (urllib, requests, etc), but that means non-HTTP uses (XMPP, Matrix, etc) will be disadvantaged.

[eighthave]

We (https://defo.ie/#contact @sftcd @jspricke @eighthave and others) have been funded again by Open Tech Fund to work on pushing ECH forward over the next 2+ years. We want to dive back in here and start implementing this. ECH support was accepted by OpenSSL as something they want to include, and our implementation is working its way through their process here: https://github.com/openssl/openssl/pull/22938

We now have more implementations to draw experience and examples from. For example, there is a client-only rustls pull request: https://github.com/rustls/rustls/pull/1718

We have a Debian PPA for installing our ECH implementations in Apache, nginx, curl, and others. So it is now easy to have ECH running on test domains: https://guardianproject.info/2023/11/10/quick-set-up-guide-for-encrypted-client-hello-ech/

And there are live domains with ECH available (https://crypto.cloudflare.com/cdn-cgi/trace), plus Firefox, Chrome and others have it on by default.

I think the easiest starting point will be to expose the OpenSSL functions, then start work on supporting the client-side with urllib and requests (https://github.com/psf/requests/issues/5972). How does that sound?