search

uazo / cromite

Cromite a Bromite fork with ad blocking and privacy enhancements; take back your browser!
https://www.cromite.org/
GNU General Public License v3.0
3.25k stars 73 forks source link

Add a flag to allow ECH even when system DNS resolver is used #1358

Open giantplaceholder opened 1 month ago

giantplaceholder commented 1 month ago

Preliminary checklist

Is your feature request related to privacy?

Yes

Is there a patch available for this feature somewhere?

Not applicable.

Describe the solution you would like

TLDR:

Describe alternatives you have considered

Not applicable.

giantplaceholder commented 1 month ago

One might check whether their browser is set up properly for ECH:

https://defo.ie/ech-check.php https://tls-ech.dev/

or by visiting Cloudflare's crypto endpoint:

https://crypto.cloudflare.com/cdn-cgi/trace

uazo commented 1 month ago

ECH needs the server's public key to work, which is provided via the RR record. if your DNS (also not doh) is able to provide it, ech is enabled in ssl connections. (see https://github.com/dadrian/ech-chrome/blob/main/explainer.md of the author of the implementation in chromium)

giantplaceholder commented 1 month ago

ECH needs the server's public key to work, which is provided via the RR record. if your DNS (also not doh) is able to provide it, ech is enabled in ssl connections. (see https://github.com/dadrian/ech-chrome/blob/main/explainer.md of the author of the implementation in chromium)

Right now my DNS is configured correctly and supplies proper answers for HTTPS RR queries:

$ dig https crypto.cloudflare.com @<LOCAL_DNS_FQDN>

; <<>> DiG 9.18.27 <<>> https crypto.cloudflare.com @<LOCAL_DNS_FQDN>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13818
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;crypto.cloudflare.com.     IN  HTTPS

;; ANSWER SECTION:
crypto.cloudflare.com.  300 IN  HTTPS   1 . alpn="http/1.1,h2" ipv4hint=162.159.137.85,162.159.138.85 ech=AEX+DQBBmQAgACBgDcqfwmAg9MZ7WiU1jFyi0UOCS8wDr7zB0FK2GjROUQAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2606:4700:7::a29f:8955,2606:4700:7::a29f:8a55

;; Query time: 133 msec
;; SERVER: <LOCAL_DNS_FQDN>#53(<LOCAL_DNS_IP>) (UDP)
;; WHEN: Wed Aug 07 11:26:35 +03 2024
;; MSG SIZE  rcvd: 204
$ dig https defo.ie @<LOCAL_DNS_FQDN>

; <<>> DiG 9.18.27 <<>> https defo.ie @<LOCAL_DNS_FQDN>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39170
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;defo.ie.           IN  HTTPS

;; ANSWER SECTION:
defo.ie.        1800    IN  HTTPS   1 . ipv4hint=213.108.108.101 ech=AED+DQA8QwAgACDggLnnbB9c2cQ57n9Y9fcJ+kEN6fgBPvzCicfmx1+eJAAEAAEAAQANY292ZXIuZGVmby5pZQAA ipv6hint=2a00:c6c0:0:116:5::10

;; Query time: 276 msec
;; SERVER: <LOCAL_DNS_FQDN>#53(<LOCAL_DNS_IP>) (UDP)
;; WHEN: Wed Aug 07 11:39:02 +03 2024
;; MSG SIZE  rcvd: 149

The server is an up-to-date PiHole, which in turn uses AdGuard's dnsproxy as an upstream, which in turn uses multiple DoH servers as its upstreams. PiHole is announced network-wide via DHCP options and communicates with clients across LAN in plain.

With that configuration, Cromite as of now does not enable ECH automatically and lacks any kind of toggle in chrome://flags to enable it manually (there used to be a "#encrypted-client-hello" flag, but it is missing both from Cromite and vanilla Chrome, both up-to-date).

Firefox, on the other hand, works with my config, after I've upgraded it to v129 and switched required flags manually (by default, it still requires DoH to be enabled in browser, but v129 now can be forced to use ECH even with plain DNS servers).

Hence, this issue.

uazo commented 1 month ago

would you like to try chromium vanilla? I need to find out if one of my patches is to blame

uazo commented 1 month ago

but it is missing both from Cromite and vanilla Chrome, both up-to-date).

ah sorry

uazo commented 1 month ago

try with chrome://net-export/ the best would be with maximum information, but I realise that there might be confidential information. maybe you check first with https://netlog-viewer.appspot.com/#import

giantplaceholder commented 1 month ago

but it is missing both from Cromite and vanilla Chrome, both up-to-date).

ah sorry

Yeah, I've just updated today to 127.0.6533.99 on vanilla Chrome.

For reference, this is what I see on at https://crypto.cloudflare.com/cdn-cgi/trace when trying to use Cromite or vanilla Chrome:

fl=<censored>
h=crypto.cloudflare.com
ip=<censored>
ts=<censored>
visit_scheme=https
uag=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=<censored>
sliver=none
http=http/2
loc=<censored>
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519Kyber768Draft00

And this is the same, but with Firefox v129 (boolean flags network.dns.native_https_query, network.dns.echconfig.enabled and network.dns.http3.echconfig.enabled all set to true). Disregard that UA is set to Tor Browser, this is intentional.

fl=<censored>
h=crypto.cloudflare.com
ip=<censored>
ts=<censored>
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; rv:102.0) Gecko/20100101 Firefox/102.0
colo=<censored>
sliver=none
http=http/2
loc=<censored>
tls=TLSv1.3
sni=encrypted
warp=off
gateway=off
rbi=off
kex=X25519Kyber768Draft00

And here's the small excrept from network event log (tested with defo.ie, which is definitely ECH enabled):

664: SOCKET
https://defo.ie <https://defo.ie same_site>
Start Time: 2024-08-07 13:15:05.245

t=4246 [st=   0] +SOCKET_ALIVE  [dt=2480+]
                  --> source_dependency = 662 (SSL_CONNECT_JOB)
t=4246 [st=   0]   +TCP_CONNECT  [dt=59]
                    --> address_list = [
                          "213.108.108.101:443"
                        ]
                    --> aliases = []
t=4246 [st=   0]      TCP_CONNECT_ATTEMPT  [dt=59]
                      --> address = "213.108.108.101:443"
t=4305 [st=  59]   -TCP_CONNECT
                    --> local_address = "<censored>:45236"
                    --> remote_address = "213.108.108.101:443"
t=4305 [st=  59]   +SSL_CONNECT  [dt=71]
t=4306 [st=  60]      SSL_HANDSHAKE_MESSAGE_SENT
                      --> bytes =
                        <TLS data removed>
                      --> type = 1
t=4306 [st=  60]      SOCKET_BYTES_SENT
                      --> byte_count = 2020
                      --> bytes =
                        <TLS data removed>
t=4366 [st= 120]      SOCKET_BYTES_RECEIVED
                      --> byte_count = 506
                      --> bytes =
                        <TLS data removed>
t=4366 [st= 120]      SSL_HANDSHAKE_MESSAGE_RECEIVED
                      --> bytes =
                        <TLS data removed>
                      --> type = 2
t=4367 [st= 121]      SSL_HANDSHAKE_MESSAGE_RECEIVED
                      --> bytes =
                       <handshake data removed>
                      --> type = 8
t=4367 [st= 121]      SSL_CERTIFICATES_RECEIVED
                      --> certificates =
                             <TLS certificates omitted>

t=4367 [st= 121]     +CERT_VERIFIER_REQUEST  [dt=8]
t=4367 [st= 121]        CERT_VERIFIER_REQUEST_BOUND_TO_JOB
                        --> source_dependency = 665 (CERT_VERIFIER_JOB)
t=4375 [st= 129]     -CERT_VERIFIER_REQUEST
t=4375 [st= 129]      SSL_HANDSHAKE_MESSAGE_RECEIVED
                      --> bytes =
                        <TLS data removed>
                      --> type = 20
t=4376 [st= 130]      SSL_HANDSHAKE_MESSAGE_SENT
                      --> bytes =
                        1<TLS data removed>
                      --> type = 20
t=4376 [st= 130]      SOCKET_BYTES_SENT
                      --> byte_count = 64
                      --> bytes =
                        <TLS data removed>
t=4376 [st= 130]   -SSL_CONNECT
                    --> cipher_suite = 4865
                    --> encrypted_client_hello = false
                    --> is_resumed = true
                    --> key_exchange_group = 29
                    --> next_proto = "http/1.1"
                    --> peer_signature_algorithm = 2052
                    --> version = "TLS 1.3"
t=4376 [st= 130]   +SOCKET_IN_USE  [dt=66]
uazo commented 1 month ago

And here's the small excrept from network event log

would you also put the one concerning the dns call?

giantplaceholder commented 1 month ago

And here's the small excrept from network event log

would you also put the one concerning the dns call?

Here you go:

t=4242 [st=0] +QUIC_SESSION_POOL_JOB  [dt=3]
               --> host = "defo.ie"
               --> network_anonymization_key = "https://defo.ie same_site"
               --> port = 443
               --> privacy_mode = "disabled"
               --> proxy_chain = "[direct://]"
t=4242 [st=0]    QUIC_SESSION_POOL_JOB_BOUND_TO
                [ --> source_dependency = 659 (HTTP_STREAM_JOB)]
t=4242 [st=0]   +HOST_RESOLVER_MANAGER_REQUEST  [dt=3]
                 --> allow_cached_response = true
                 --> dns_query_type = "UNSPECIFIED"
                 --> host = "https://defo.ie"
                 --> is_speculative = false
                 --> network_anonymization_key = "https://defo.ie same_site"
                 --> secure_dns_policy = 0
t=4242 [st=0]      HOST_RESOLVER_MANAGER_IPV6_REACHABILITY_CHECK
                   --> cached = true
                   --> ipv6_available = false
t=4242 [st=0]      HOST_RESOLVER_MANAGER_CREATE_JOB
t=4242 [st=0]      HOST_RESOLVER_MANAGER_JOB_ATTACH
                  [ --> source_dependency = 661 (HOST_RESOLVER_IMPL_JOB)]
t=4245 [st=3]   -HOST_RESOLVER_MANAGER_REQUEST
t=4245 [st=3] -QUIC_SESSION_POOL_JOB
t=4242 [st=0] +HOST_RESOLVER_MANAGER_JOB  [dt=3]
               --> dns_query_types = [
                     "A",
                     "AAAA",
                     "HTTPS"
                   ]
               --> host = "https://defo.ie"
               --> network_anonymization_key = "https://defo.ie same_site"
               --> secure_dns_mode = 0
              [ --> source_dependency = 660 (QUIC_SESSION_POOL_DIRECT_JOB)]
t=4242 [st=0]    HOST_RESOLVER_MANAGER_JOB_REQUEST_ATTACH
                 --> priority = "HIGHEST"
                [ --> source_dependency = 660 (QUIC_SESSION_POOL_DIRECT_JOB)]
t=4242 [st=0]    HOST_RESOLVER_MANAGER_JOB_STARTED
t=4242 [st=0]   +HOST_RESOLVER_SYSTEM_TASK  [dt=3]
t=4242 [st=0]      HOST_RESOLVER_MANAGER_ATTEMPT_STARTED
                   --> attempt_number = 1
t=4245 [st=3]   -HOST_RESOLVER_SYSTEM_TASK
                 --> address_list = [
                       "213.108.108.101:0",
                       "[2a00:c6c0:0:116:5::10]:0"
                     ]
                 --> aliases = []
t=4245 [st=3]    HOST_RESOLVER_MANAGER_ATTEMPT_FINISHED
                 --> attempt_number = 1
t=4245 [st=3] -HOST_RESOLVER_MANAGER_JOB
uazo commented 1 month ago

now would you do one with a call to a doh server? I am only interested in the section HOST_RESOLVER_MANAGER_JOB. thanks

giantplaceholder commented 1 month ago

This is with Cloudflare set up as DoH upstream. Seems that when secure_dns_mode is set to zero (as it would when "Secure DNS" is disabled and provider is not selected explicitly) it doesn't even tries to lift data from HTTP RR and proceeds with plain SNI.

t=1757 [st=  0] +HOST_RESOLVER_MANAGER_JOB  [dt=279]
                 --> dns_query_types = [
                       "A",
                       "HTTPS"
                     ]
                 --> host = "https://defo.ie"
                 --> network_anonymization_key = "https://defo.ie same_site"
                 --> secure_dns_mode = 2
                 --> source_dependency = 663 (QUIC_SESSION_POOL_DIRECT_JOB)
t=1757 [st=  0]    HOST_RESOLVER_MANAGER_JOB_REQUEST_ATTACH
                   --> priority = "HIGHEST"
                   --> source_dependency = 663 (QUIC_SESSION_POOL_DIRECT_JOB)
t=1757 [st=  0]    HOST_RESOLVER_MANAGER_JOB_STARTED
t=1757 [st=  0]   +HOST_RESOLVER_DNS_TASK  [dt=279]
                   --> secure = true
                   --> transactions_needed = [
                         {
                           "dns_query_type": "A"
                         },
                         {
                           "dns_query_type": "HTTPS"
                         }
                       ]
t=1757 [st=  0]      DNS_TRANSACTION
                     --> source_dependency = 665 (DNS_TRANSACTION)
t=1757 [st=  0]      DNS_TRANSACTION
                     --> source_dependency = 667 (DNS_TRANSACTION)
t=2036 [st=279]      HOST_RESOLVER_DNS_TASK_EXTRACTION_RESULTS
                     --> results = [
                           {
                             "domain_name": "defo.ie",
                             "endpoints": [
                               {
                                 "address": "213.108.108.101",
                                 "port": 0
                               }
                             ],
                             "hosts": [],
                             "query_type": "A",
                             "source": "dns",
                             "strings": [],
                             "timed_expiration": "13367502085755043",
                             "type": "data"
                           }
                         ]
t=2036 [st=279]      HOST_RESOLVER_DNS_TASK_EXTRACTION_RESULTS
                     --> results = [
                           {
                             "domain_name": "defo.ie",
                             "metadatas": [
                               {
                                 "metadata_value": {
                                   "ech_config_list": "AED+DQA8cQAgACDhjTYKG/y12zAhLcm/PZwrBbPFqeH8FORsbiAZDD7aWwAEAAEAAQANY292ZXIuZGVmby5pZQAA",
                                   "supported_protocol_alpns": [
                                     "http/1.1"
                                   ],
                                   "target_name": "defo.ie"
                                 },
                                 "metadata_weight": 1
                               }
                             ],
                             "query_type": "HTTPS",
                             "source": "dns",
                             "timed_expiration": "13367500285755606",
                             "type": "metadata"
                           }
                         ]
t=2036 [st=279]   -HOST_RESOLVER_DNS_TASK
                   --> results = {
                         "aliases": [
                           "defo.ie"
                         ],
                         "canonical_names": [
                           "defo.ie"
                         ],
                         "endpoint_metadatas": [
                           {
                             "endpoint_metadata_value": {
                               "ech_config_list": "AED+DQA8cQAgACDhjTYKG/y12zAhLcm/PZwrBbPFqeH8FORsbiAZDD7aWwAEAAEAAQANY292ZXIuZGVmby5pZQAA",
                               "supported_protocol_alpns": [
                                 "http/1.1"
                               ],
                               "target_name": "defo.ie"
                             },
                             "endpoint_metadata_weight": 1
                           }
                         ],
                         "expiration": "13367482409075070",
                         "host_ports": [],
                         "hostname_results": [],
                         "ip_endpoints": [
                           {
                             "endpoint_address": "213.108.108.101",
                             "endpoint_port": 0
                           }
                         ],
                         "text_records": []
                       }
t=2036 [st=279] -HOST_RESOLVER_MANAGER_JOB
uazo commented 1 month ago

in the code I found this:

doc refers to the flag kUseDnsHttpsSvcbEnableInsecure which is, however, no longer present in the code but I found in features.cc which should be v105.

I will try to check what that flag was doing and especially why it was removed... after my holidays! :)

giantplaceholder commented 1 month ago

kUseDnsHttpsSvcbEnableInsecure

It would be great if you'd take a look. Thanks!

uazo commented 1 week ago

first of all, are you talking about android? I looked at the code, the flag is no longer present but linked to [isPrivateDnsActive](https://developer.android.com/reference/android/net/LinkProperties#isPrivateDnsActive()) API.

Basically, it appears that if the system DNS is set to ‘private’, the chromium dns client does not activate the HTTPS type request (https://issues.chromium.org/issues/40219327#comment13). can you confirm this for me?

behaviour changes depending on the version of Android, which one are you using?

giantplaceholder commented 1 week ago

first of all, are you talking about android? I looked at the code, the flag is no longer present but linked to [isPrivateDnsActive](https://developer.android.com/reference/android/net/LinkProperties#isPrivateDnsActive()) API.

Basically, it appears that if the system DNS is set to ‘private’, the chromium dns client does not activate the HTTPS type request (https://issues.chromium.org/issues/40219327#comment13). can you confirm this for me?

behaviour changes depending on the version of Android, which one are you using?

I have no idea how it behaves on Android, as I do not use Chromium-based browsers on my handheld.

The behavior I described happens on desktop Linux and desktop Windows builds, taken off the local github CI.

uazo commented 1 week ago

for desktops there is a policy to be activated: https://chromeenterprise.google/policies/#AdditionalDnsQueryTypesEnabled

ref: https://issues.chromium.org/issues/40763530