Is this updated to work on cloudflare servers?

appkoders commented 3 years ago

I tried to compile this and the esnistuff I followed the instructions here https://defo.ie/building-curl-openssl-with-esni.html

I tried to test it like the instructions from the link above

$ ./testclient.sh -H ietf.org
Running ./testclient.sh at 20190828-072413
./testclient.sh Summary: 
Looks like 1 ok's and 0 bad's.

But I only get

Running ./testclient.sh at 20210427-025718
./testclient.sh Summary:
Looks like 0 ok's and 0 bad's.

I got 0 ok's

sftcd commented 3 years ago

On 27/04/2021 03:59, AppKoders wrote:

I tried to compile this and the esnistuff I followed the instructions here https://defo.ie/building-curl-openssl-with-esni.html

Ah sorry. Outdated instructions for curl.

Will update and let you know,

I tried to test it like the instructions from the link above

$ ./testclient.sh -H ietf.org
Running ./testclient.sh at 20190828-072413
./testclient.sh Summary:
Looks like 1 ok's and 0 bad's.

But I only get

Running ./testclient.sh at 20210427-025718
./testclient.sh Summary:
Looks like 0 ok's and 0 bad's.

I got 0 ok's

sftcd commented 3 years ago

Hiya,

I've updated things variously so if you'd like to start over following the HOWTO at [1] and let me know that'd be great,

S.

[1] https://github.com/sftcd/openssl/blob/ECH-without-ESNI/esnistuff/building-curl-openssl-with-ech.md

sftcd commented 3 years ago

Closing - the ECH_UPFRONT_DEC branch works against cloudflare at draft-10 level. Will be updated to draft-11 in the nearish future.

calvin2021y commented 2 years ago

https://github.com/niallor/curl/issues/2

I can not made ESNI work, confirm it work on firefox.

sftcd commented 2 years ago

Hiya,

On 13/07/2021 08:17, calvin2021y wrote:

https://github.com/niallor/curl/issues/2

I can not made ESNI work, confirm it work on firefox.

About a year ago Firefox had support for ESNI (draft-02 of the spec). Things have moved along since and they removed that. We're now working on draft-10 of the spec (and have renamed ESNI to ECH). Mozilla people are working on their implementation so there should be support at some point in the not too distant future.

Cheers, S.

calvin2021y commented 2 years ago

If you download https://support.mozilla.org/en-US/kb/switch-to-firefox-extended-support-release-esr , and enable ESNI from about:config, then you are ready to go with ESNI from all cloudflare website.

The ECH support maybe in near future, but right now we have to rely on ESNI to avoid SNI plaintext.

I am sure sftcd/openssl has work before, but right now it can not establish connection with cloudflare or only.esni.defo.ie.

https://github.com/OperatorFoundation/meekheavy_proof_of_concept include a static library, I try download this return error (also try cloudflare from this project).

SessionID: 8F443792B124DA0BAB737F6B23EE9328
sessionheader::  X-Session-Id: 8F443792B124DA0BAB737F6B23EE9328
* STATE: INIT => CONNECT handle 0x557be63c7208; line 1368 (connection #-5000)
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => WAITRESOLVE handle 0x557be63c7208; line 1409 (connection #0)
*   Trying 104.16.123.96:443...
* TCP_NODELAY set
* STATE: WAITRESOLVE => WAITCONNECT handle 0x557be63c7208; line 1488 (connection #0)
* Connected to www.cloudflare.com (104.16.123.96) port 443 (#0)
* STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x557be63c7208; line 1544 (connection #0)
* Marked for [keep alive]: HTTP default
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* Found ESNI parameters:
*   flag ssl_enable_esni (SET)
*   flag ssl_strict_esni (SET)
*   STRING_ESNI_SERVER (www.cloudflare.com)
*   STRING_ESNI_COVER (cloudflare.com)
*   STRING_ESNI_ASCIIRR (/wEnjRX3ACQAHQAgMwO3RbaOyBs9kG+rLS4/Lz8CSFIvM55atQ4KGvOashcAAhMBAQQAAAAAYOlTgAAAAABg8TyAAAA=)
* SSL_ESNI object version (ff01)
* Found 1 ESNI key
* Configured encrypted server name (ESNI) TLS extension
* STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x557be63c7208; line 1559 (connection #0)
* error:140000EA:SSL routines::callback failed
* Marked for [closure]: Failed HTTPS connection
* multi_done
* Closing connection 0
* The cache now contains 0 members
* Expire cleared (transfer 0x557be63c7208)
curl_easy_perform() failed: SSL connect error

If this can be fixed we can still use it until cloudflare deploy ECH to all site.

sftcd commented 2 years ago

On 14/07/2021 06:23, calvin2021y wrote:

If you download https://support.mozilla.org/en-US/kb/switch-to-firefox-extended-support-release-esr , and enable ESNI from about:config, then you are ready to go with ESNI from all cloudflare website.

Ah. I didn't know that was still available. I'll check it out later and get back.

S.

The ECH support maybe in near future, but right now we has rely on ESNI to avoid SNI plaintext.

I am sure sftcd/openssl have work before, but right now it can not establish connection with cloudflare or only.esni.defo.ie.

https://github.com/OperatorFoundation/meekheavy_proof_of_concept include a static library, I try download this return error (also try cloudflare from this project).

SessionID: 8F443792B124DA0BAB737F6B23EE9328
sessionheader::  X-Session-Id: 8F443792B124DA0BAB737F6B23EE9328
* STATE: INIT => CONNECT handle 0x557be63c7208; line 1368 (connection #-5000)
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => WAITRESOLVE handle 0x557be63c7208; line 1409 (connection #0)
*   Trying 104.16.123.96:443...
* TCP_NODELAY set
* STATE: WAITRESOLVE => WAITCONNECT handle 0x557be63c7208; line 1488 (connection #0)
* Connected to www.cloudflare.com (104.16.123.96) port 443 (#0)
* STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x557be63c7208; line 1544 (connection #0)
* Marked for [keep alive]: HTTP default
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
   CApath: none
* Found ESNI parameters:
*   flag ssl_enable_esni (SET)
*   flag ssl_strict_esni (SET)
*   STRING_ESNI_SERVER (www.cloudflare.com)
*   STRING_ESNI_COVER (cloudflare.com)
*   STRING_ESNI_ASCIIRR (/wEnjRX3ACQAHQAgMwO3RbaOyBs9kG+rLS4/Lz8CSFIvM55atQ4KGvOashcAAhMBAQQAAAAAYOlTgAAAAABg8TyAAAA=)
* SSL_ESNI object version (ff01)
* Found 1 ESNI key
* Configured encrypted server name (ESNI) TLS extension
* STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x557be63c7208; line 1559 (connection #0)
* error:140000EA:SSL routines::callback failed
* Marked for [closure]: Failed HTTPS connection
* multi_done
* Closing connection 0
* The cache now contains 0 members
* Expire cleared (transfer 0x557be63c7208)
curl_easy_perform() failed: SSL connect error

so if this can be fixed we can still use it until cloudflare deploy ECH.

calvin2021y commented 2 years ago

A tip to enbale ESNI on firefox ESR:

network.security.esni.enabled=true
network.trr.mode=3

Restart firefox then visit https://www.cloudflare.com/ssl/encrypted-sni/ or https://www.cloudflare.com/cdn-cgi/trace .

calvin2021y commented 2 years ago

any update on this ?

sftcd commented 2 years ago

Sorry - I forgot;-( But I took a look today. So yes the FF ESR version still "passes" the CF page check and also connects fine still to e.g. https://canbe.esni.defo.ie. I also re-built my last OpenSSL build that had ESNI and fixed an ESNI-specific bug in that just now. After that, I can use the testclient.sh script to use ESNI with both CF and with canbe.esni.defo.ie. Note that with CF I have to omit the cleartext SNI for it to work for some reason. (That usen't be the case.) So if you built that version of the code in $HOME/openssl-esni then the command line to try after building is:

$ cd esnistuff
$ TOP=.. ./testclient.sh -H www.cloudflare.com -dv -c NONE
...lots of output...
ESNI: success: clear sni: 'none', hidden: 'www.cloudflare.com'

Two other notes:

I would not recommend trying ECH with that build - it's early draft-10 stuff and the latest is about to move on to draft-13 (and I don't intend to devote effort to earlier versions of ECH)
It looks like there's a leak when doing the above, sorry - don't have any more time for this old stuff today but feel free to try fix and get back with a PR or whatever.

calvin2021y commented 2 years ago

Thanks very much, I verify this work without cover string.

I am not able to locate the leak position, will give it more try.

sftcd commented 2 years ago

On 14/08/2021 13:55, calvin2021y wrote:

Thanks very much, I verify this work without cover string.

I am not able to locate the leak position, will give it more try.

Valgrind will show where. The "esnistuff/testclient.sh" script can use that for example.

Cheers, S.

sftcd / openssl

Is this updated to work on cloudflare servers? #15