search

refraction-networking / utls

Fork of the Go standard TLS library, providing low-level access to the ClientHello for mimicry purposes.
BSD 3-Clause "New" or "Revised" License
1.67k stars 237 forks source link

How to implement the "extensionEncryptedClientHello (boringssl) (65037)," extended #240

Open xiaoweigege opened 1 year ago

xiaoweigege commented 1 year ago

How to implement the "extensionEncryptedClientHello (boringssl) (65037)," extended

jjsaunier commented 1 year ago

You can take a look at https://github.com/cloudflare/go/commit/9ea1834ecef8601872a7a90e19d31a2165537e36 and then

xiaoweigege commented 1 year ago

There doesn't seem to be a corresponding implementation

jjsaunier commented 1 year ago

"extensionEncryptedClientHello (boringssl) (65037)," extended

https://github.com/cloudflare/go/commit/9ea1834ecef8601872a7a90e19d31a2165537e36#diff-f93ae351e135c874558cc63997db12a4c2c4d220f3bb2be84ecf6c808b2b9599R126

0xfe0d is 65037 in decimal, and ECH is the abbreviation of EncryptedClientHello, why are you saying it does not correspond?

xiaoweigege commented 1 year ago

Thank you, I see! Does adding this extension allow for faster validation of akamai waf?

jjsaunier commented 1 year ago

Probably not; I don't know a browser that has implemented it yet (if it does, it's behind a feature flag), and the RFC is still in draft, so I doubt they use it as a signal or give a faster validation since 99% of traffic do not have it

gaukas commented 1 year ago

a browser that has implemented it yet

the latest Firefox is said to be shipped with ECH enable iff DNS-over-HTTPS is correctly configured.

gaukas commented 1 year ago

But yeah it is probably a great idea to implement ECH for uTLS. PR is welcome.

xiaoweigege commented 1 year ago

Can be like "FakeDelegatedCredentialsExtension", implement a placeholder. It is not clear how ECH is implemented

gaukas commented 1 year ago

how ECH is implemented

I believe that the cloudflare/go actually have it implemented in full. If interested someone may just port it. The License should be compatible.

xiaoweigege commented 1 year ago

Can you transplant him? I am not familiar with golang at the moment

gaukas commented 1 year ago

In case you are asking if I (or anyone from the refraction team) will port this feature from cloudflare, the answer is no, currently we are all at capacity and adding an optional feature is not prioritized.

But if anyone from the community would like to open a pull request on that, I will be happy to review it.

jjsaunier commented 1 year ago

Cloudflare make a blog post about it

https://blog.cloudflare.com/announcing-encrypted-client-hello/ with browser support upcoming

gaukas commented 1 year ago

It might be a good time to start discussing how significant is it for uTLS to support ECH and what does a sense-making configurable interface look like in uTLS for ECH.

pitagoras530 commented 10 months ago

hi, is there any update to this extension? Most sites do cause issues, if you do not provide the extension

gaukas commented 10 months ago

Most sites do cause issues, if you do not provide the extension.

Would you like to elaborate? Which sites cause issues and what kinds of issue.

My impression is that none of the TLS extensions should be mandatory. Especially a draft extension in no case should be required in anyways.

So far the work on this extension is not prioritized, given that it is not yet generically available, plus it is not yet solidified into an IETF standard. But again, we are open to pull requests adding new extensions.

gaukas commented 10 months ago

And here's my 2 cents on implementing extensionEncryptedClientHello:

I still believe there are good reason to hold onto this feature even though cloudflare might already have a readily available implementation, until ECH is fairly popular among TLS servers (instead of just the top 2 web browsers, which are considered TLS clients).

Once uTLS, or any popular circumvention tool started exploiting ECH to circumvent DPI-based blocking, censors may respond by blocking/disabling it completely with little to no collateral damage.

Btw we already saw reports showing some strong censors blocking the DNS-over-HTTPS, which is usually considered a prerequisite of successful ECH bootstrapping.

gaukas commented 10 months ago

And I am open to merging changes from the hard fork on cloudflare/go by cloudflare , maybe with necessary modification/changes. The LICENSE looks compatible to me (BSD-3-Clause) and their implementation has a fairly assured quality.

I haven't looked into how much additional work will be required on uTLS to support ECH, but it shouldn't be too bad comparing to supporting PSK and post-quantum key share (kyber768-ed25519) in uTLS, which has been handled with a satisfactory standard thanks to the community.

pitagoras530 commented 10 months ago

And I am open to merging changes from the hard fork on cloudflare/go by cloudflare , maybe with necessary modification/changes. The LICENSE looks compatible to me (BSD-3-Clause) and their implementation has a fairly assured quality.

I haven't looked into how much additional work will be required on uTLS to support ECH, but it shouldn't be too bad comparing to supporting PSK and post-quantum key share (kyber768-ed25519) in uTLS, which has been handled with a satisfactory standard thanks to the community.

In order to replicate the Ja3 of google chrome, you need the ECH extension else most cloudflare sites block you based on that.

gaukas commented 10 months ago

you need the ECH extension else most cloudflare sites block you based on that

If I understand correctly: when you advertise your support of ECH using the GenericExtension, cloudflare will not be able to complete the handshake correctly. It is pretty much expected. But are there any compelling reason to not just simply give up advertising it? i.e., why do you want to parrot this version of Google Chrome.

The "needs for parrot" again leads to my concern: how popular is the ECH extension on TLS clients and how popular is ECH support on TLS servers? Without an assuring answer, I am afraid implementing ECH too soon in circumvention community will lead to the minimal collateral damage when trivially blocking ECH. The best bet is for us to we wait until ECH become a default behavior for the vast majority of benign TLS clients/servers, so it doesn't become a strong fingerprintable feature.

biaosheng commented 10 months ago

Currently, Google Chrome 120 version has added ECH

gaukas commented 10 months ago

Currently, Google Chrome 120 version has added ECH

Could you please double check if Chrome 120 sends a greased ECH extension if prerequisite for ECH is not met? i.e., if no DoH configured, if target domain is not configured to support ECH, etc.

gaukas commented 9 months ago

So as the first step, today we merged #266 to add the GREASE ECH extension. For pure parroting purposes, this should suffice.

We will keep holding onto the full ECH implementation until fully ECH is fairly common...

Now available in v1.6.0, cheers🥂

biaosheng commented 9 months ago

So as the first step, today we merged #266 to add the GREASE ECH extension. For pure parroting purposes, this should suffice.

We will keep holding onto the full ECH implementation until fully ECH is fairly common...

Now available in v1.6.0, cheers🥂

Wow, you are so great!