search

saucesteals / utlsproxy

MITM Proxy with TLS mimicry
GNU General Public License v3.0
33 stars 4 forks source link

server sent ECH retry configs without client sending ECH extension #9

Open ngehrsitz opened 3 months ago

ngehrsitz commented 3 months ago

Hi,

I am trying to intercept the traffic from a website that embeds code from https://unpkg.com/ but that fails with the following error:

2024/06/07 22:43:25 [007] WARN: Cannot handshake to unpkg.com:443
%!(EXTRA *errors.errorString=tls: server sent ECH retry configs without client sending ECH extension)

I am using Firefox 125.0.3 as the client. I already tried toggling network.dns.echconfig.enabled and network.dns.use_https_rr_as_altsvc but that didn´t change anything. There is a specific ECH test site, that fails to load with the same error: https://defo.ie/ech-check.php In the debugger I can see that the ECH extension is indeed missing image The ECH support was implemented in https://github.com/refraction-networking/utls/issues/240, but I couldn´t find an issue with that code. Could it be that the clientHello is not interpreted correctly?

dr3adx commented 3 months ago

Hi,

I am trying to intercept the traffic from a website that embeds code from https://unpkg.com/ but that fails with the following error:

2024/06/07 22:43:25 [007] WARN: Cannot handshake to unpkg.com:443
%!(EXTRA *errors.errorString=tls: server sent ECH retry configs without client sending ECH extension)

I am using Firefox 125.0.3 as the client. I already tried toggling network.dns.echconfig.enabled and network.dns.use_https_rr_as_altsvc but that didn´t change anything. There is a specific ECH test site, that fails to load with the same error: https://defo.ie/ech-check.php In the debugger I can see that the ECH extension is indeed missing image The ECH support was implemented in refraction-networking/utls#240, but I couldn´t find an issue with that code. Could it be that the clientHello is not interpreted correctly?

same error here, did you find a solution? throw me your discord if u have it

ngehrsitz commented 3 months ago

No, I didn´t find a solution. I think I will try to implement my MitM use-case with a browser plugin instead

clouedoc commented 3 months ago

Maybe our client is too strict?

Maybe you could just comment out a specific part of the code to be less strict when it comes to receiving an ECH retry config without having sent an extension.

https://guardianproject.info/2021/11/30/implementing-tls-encrypted-client-hello/

image

--> our client is not too strict. Rather, it's misbehaving — the target doesn't understand what we're sending so it resorts to an "ECH Retry"

Erratum. It looks like the target server has trouble decrypting the data?

Okay, then, maybe it's just Firefox's ClientHello trigerring a bug?

It would be worthwhile to check if the problem arises with a different client.

E.g. can you access the same website with cURL?

On my side:

image
  1. cURL+utlsproxy with https://unpkg.com/ works fine.
  2. cURL+utlsproxy with https://defo.ie/ech-check.php hangs. Maybe I should try waiting a bit more 👀
  3. cURL without utlsproxy with https://defo.ie/ech-check.php works fine

I didn't try Firefox.

It looks like Firefox's ClientHello is not the entire problem. Firefox might be triggering additionnal problems making it impossible to access unpkg. It's already possible to reproduce problems with cURL, but only with https://defo.ie/ech-check.php

Not sure what to conclude.

--> The first step in resolving the issue is to get cURL+utlsproxy+https://defo.ie/ech-check.php working.

Wireshark capture

Capture: curl+utlsproxy

I did the following:

  1. Start a Wireshark capture
  2. Start utlsproxy
  3. Send a request to defo.ie
  4. Wait a few seconds (the request is still hanging)
  5. Stop the Wireshark capture
  6. Kill utlsproxy
image

image

defo-ie-curl-utlsproxy.pcapng.zip

Capture: cURL alone

image

defo-ie-curl-standalone.pcapng.zip

Comparaison

(left is utlsproxy+cURL, right is cURL standalone)

We can check the flow of TLS packet content types:

image

We can notice that there is a difference there already.

  1. CLIENT: Handshake
  2. SERVER: Handshake, Change Cipher Spec (additionnal Handshake in the utlsproxy configuration)

Since we can already see a difference at packet 2, I bet that there is something different in packet 1 beside the content type. (there is also the hypothesis that I'm just missing some TLS keys to decrypt some parts of the flow)

Erratum: there's no difference

image

So, the issue is not in the first Handshake packet (Client Hello).

We can skip packets sent by the server (2 and 3), assuming that they are the same and we just cannot decrypt some part of them without configuring cURL to output an NSS Key Log.

We can see that utlsproxy finalizes the handshake maybe a bit early on the left? (reminder: utlsproxy+cURL on the left, curl alone on the right)

image

Not sure what's going on.

I think that to resolve the issue, the best would be to perform the following steps:

  1. Get a cURL capture with NSS Key Log enabled: https://everything.curl.dev/usingcurl/tls/sslkeylogfile.html
  2. Do another comparaison having the SSL keys

I'm going to do something else with my day, but what I would advise to do to resolve the issue:

  1. Get an utlsproxy + cURL capture, export the TLS packets only in a Wireshark file, embed the SSLKeylogs using ITS, following this tutorial: https://wiki.wireshark.org/TLS#embedding-decryption-secrets-in-a-pcapng-file
  2. Same for a cURL capture
  3. Post the captures here
  4. Compare them until we find the exact problem.
saucesteals commented 3 months ago

Hello everyone! Should be fixed on latest main branch now. Let me know

ngehrsitz commented 3 months ago

I just did a quick test with https://unpkg.com/ on 44a0906ecb39a27ba3183719cdfc423f2bfe9450 and that works fine. But https://defo.ie/ech-check.php still refuses to work image I can also see the same error with regular domains like https://api.mapbox.com/ so the issue is not just limited to the test site.