search

ooni / probe

OONI Probe network measurement tool for detecting internet censorship
https://ooni.org/install
BSD 3-Clause "New" or "Revised" License
759 stars 142 forks source link

add probe for TLSv1.3 ECH (Encrypted ClientHello, ESNI replacement) #1453

Open eighthave opened 3 years ago

eighthave commented 3 years ago

I'm working with @sftcd on implementing the TLSv1.3 ECH standard as widely as possible. One thing that would be quite helpful would be to have data on whether anyone is blocking ECH currently. OONI seems like the best platform for that. I've never implemented an OONI probe. I have some funding to pay either myself or someone more qualified to do this.

I believe there is already an ESNI probe so that would be quite closely related.

bassosimone commented 3 years ago

@eighthave that's super interesting! If I understand it correctly, your project/implementation is to add support for that in OpenSSL, am I correct? We are currently mostly using Go code. I know https://github.com/cloudflare/go is a fork of Go that also implements ECH. Could using this be a (maybe not so cheap but certainly quite interesting) way of running these tests with OONI?

eighthave commented 3 years ago

Yes, we have implemented it in openssl, and are following the boringssl implementation. Then we also implement it in clients like curl, lighttpd, nginx, etc.

It looks like that fork implements ECH already: https://github.com/cloudflare/go/blob/b0875fbf17405ccd0e3a57ec9fa0614d5e9be13c/src/crypto/tls/ech_test.go

@sftcd do you know about how current Cloudflare's Go implementation is?

sftcd commented 3 years ago

On 20/04/2021 17:01, Hans-Christoph Steiner wrote:

@sftcd do you know about how current Cloudflare's Go implementation is?

CF are up to date with draft-10.

But for ooni you don't need a real ECH just to attempt with a GREASE'd one, so you just need the code-point and about the right number of random octets.

S.

eighthave commented 3 years ago

I guess the tricky part is getting access to the TLS transaction at the right place to insert that code-point and random octets

sftcd commented 3 years ago

On 20/04/2021 17:57, Hans-Christoph Steiner wrote:

I guess the tricky part is getting access to the TLS transaction at the right place to insert that code-point and random octets

Or just use CF's go implementation?

If some specially crafted something were needed I'd be happy to try help with that, but I'd hope that's not the case.

S.

eighthave commented 3 years ago

I've never done any Go before, so I'm not a great candidate to code this. @bassosimone how can I help make this happen?

eighthave commented 3 years ago

This is still interesting to us. Anything we can do to help? We have lots of example code in C and C++ for example.

d1vyank commented 2 years ago

I am interested in taking this up. Given ECH’s ‘do not stick out’ requirements and attempts by censors to block prior iterations of the protocol (ESNI), I think it is important to test early on if the draft version of ECH is subject to any network interference.

I propose to add an experimental test to probe-cli or miniooni that attempts a TLS connection with an encrypted Client Hello. I’ve looked into the two approaches discussed above:

I am leaning towards the GREASE approach as it seems like a less intrusive change than using a Go fork. Even though GREASE is not a complete ECH connection, it will provide valuable early evidence of interference to TLS connections using the ECH extension.

Happy to hear your thoughts!

sftcd commented 2 years ago

On 29/07/2022 07:38, Divyank Katira wrote:

I am leaning towards the GREASE approach as it seems like a less intrusive change than using a Go fork. Even though GREASE is not a complete ECH connection, it will provide valuable early evidence of interference to TLS connections using the ECH extension.

That makes sense. Happy to provide samples etc if that helps and/or if you're testing against defo.ie, I can look at logs if debugging's needed.

One suggestion - when you GREASE you should expect to get a retry_config in return that has a (or some) ECHConfig(s) that the server uses. It'd be interesting to record those and see if they differ from locale to locale perhaps. That'd be most interesting as help in figuring out any interop problems and likely less so in terms of exploring censorship as you should only see the retry_config after the server's authenticated.

Happy to discuss more as you get to doing it.

Cheers, S.

kelmenhorst commented 1 year ago

Update: Since we have ECH check in the tree for a few months now, we want to start collecting the first baseline measurements. We will start by adding it to the experimental nettest suite, with single static input.

eighthave commented 1 year ago

Great! Let me know if you need anything from us (DEfO). For example, we can recommend some sites to monitor, based on your parameters.

kelmenhorst commented 1 year ago

@eighthave Thanks, that would be great, what sites would you recommend then?

eighthave commented 1 year ago

https://crypto.cloudflare.com/cdn-cgi/trace and tls-ech.dev would probably be a good place to start.

we maintain a list here, you could try any of those https://github.com/sftcd/openssl/blob/ECH-draft-13c/esnistuff/smoke_ech.sh#L10

eighthave commented 11 months ago

We just published a HOWTO for how to set up dev/test servers using our TLS Encrypted ClientHello (ECH) enabled forks of OpenSSL, nginx and curl running on Debian. It should be very quick to get started using a new domain: https://guardianproject.info/2023/11/10/quick-set-up-guide-for-encrypted-client-hello-ech/

We just bought some dirt cheap domains and set up test sites. I think OONI devs will find this useful going forward. We might be able to provide a pre-configured VPS for you to mess with too.

eighthave commented 6 months ago

Just checking in on this. I see some things have been merged, but I can't find any references to ECH in https://explorer.ooni.org. Is there a way to see data on ECH yet?

Also, we have developer time and budget to spend on supporting OONI with this. Please let us know how best to jump in here.

eighthave commented 5 months ago

IIRC, these tests are written in Go. golang's core crypto/tls merged client ECH support, should be in golang 1.23. https://go-review.googlesource.com/c/go/+/578575

eighthave commented 5 days ago

We want to push this forward. One very useful improvement would be to save the RetryRequest after sending ECH GREASE. A full ECH connection with DNS tracking is the ideal.

FYI @irl has joined us in working on DEfO, and pointed us to the OONI ECH spec and other things in this codebase.

If you're on Matrix, we can help you interactively with any ECH questions or problems: https://matrix.to/#/#ech-dev:matrix.org

Go 1.23 has partial support for ECH on the Client Side: https://github.com/golang/go/issues/63369. It still cannot fetch the ECH keys and there is no server-side support, but they would like to add it to 1.24: https://github.com/golang/go/issues/68500 The client takes bytes for the config list: https://pkg.go.dev/crypto/tls@master#Config.EncryptedClientHelloConfigList

sftcd commented 5 days ago

RetryRequest

Just to clarify: that should be "retry_configs" - in the ECH protocol, if a client sends GREASE'd ECH then a server that does support ECH ought reply including an ECH extension in it's EncryptedExtensions reply with the payload of that being a set of ECHConfig values that can be used with that server (i.e. the stuff a browser would have gotten from an HTTPS RR in the DNS). If the OONI probe could record those values that would be useful data. (I'm not sure what golang APIs might allow doing that, but they may exist.) Going further, if the OONI probe could then use those values in a real ECH attempt (i.e. non-GREASE) and record that result, then that'd be a fine 2nd step forward once the golang TLS library has support for the client-side of ECH, and without having to code up all the HTTPS RR DNS handling.