Add a flag to allow ECH even when system DNS resolver is used

giantplaceholder commented 3 months ago

Preliminary checklist

[X] I have read the README
[X] I have searched existing issues for my feature request. This is a new issue (NOT a duplicate) and is not related to another issue.
[X] I have searched wont fix issues and this request is not among them
[X] This is a feature request for the Cromite browser; not the website nor F-Droid nor anything else.

Is your feature request related to privacy?

Yes

Is there a patch available for this feature somewhere?

Not applicable.

Describe the solution you would like

TLDR:

Chromium supports ECH (Encrypted Client Hello) since v117 (see https://chromestatus.com/feature/6196703843581952)
As of now, it works only when DoH is enabled in browser (the reasoning is that DNS' replies should be trustworthy)
I don't want to use DNS from Cloudflare, Google and whomever else Google put there as default options - they have enough data on me as it is.
The aforementioned effectively makes ECH unusable for those whom run their own DNS on their LAN that use DoH/DoT upstreams of user's choice, instead of using DoH resolvers from big data baked into the browser. This also makes it harder to implement network-wide DNS-based ad blocking - you'll have to set up a local DoH proxy with a valid TLS certificate after your PiHole for example.
I'd like to see a feature flag or configuration toggle that will force-enable ECH even for those configurations when DoH ("Secure DNS" toggle) is disabled in browser. I trust my own home-cooked DNS enough.

Describe alternatives you have considered

Not applicable.

giantplaceholder commented 3 months ago

One might check whether their browser is set up properly for ECH:

https://defo.ie/ech-check.php https://tls-ech.dev/

or by visiting Cloudflare's crypto endpoint:

https://crypto.cloudflare.com/cdn-cgi/trace

uazo commented 3 months ago

ECH needs the server's public key to work, which is provided via the RR record. if your DNS (also not doh) is able to provide it, ech is enabled in ssl connections. (see https://github.com/dadrian/ech-chrome/blob/main/explainer.md of the author of the implementation in chromium)

giantplaceholder commented 3 months ago

ECH needs the server's public key to work, which is provided via the RR record. if your DNS (also not doh) is able to provide it, ech is enabled in ssl connections. (see https://github.com/dadrian/ech-chrome/blob/main/explainer.md of the author of the implementation in chromium)

Right now my DNS is configured correctly and supplies proper answers for HTTPS RR queries:

$ dig https crypto.cloudflare.com @<LOCAL_DNS_FQDN>

; <<>> DiG 9.18.27 <<>> https crypto.cloudflare.com @<LOCAL_DNS_FQDN>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13818
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;crypto.cloudflare.com.     IN  HTTPS

;; ANSWER SECTION:
crypto.cloudflare.com.  300 IN  HTTPS   1 . alpn="http/1.1,h2" ipv4hint=162.159.137.85,162.159.138.85 ech=AEX+DQBBmQAgACBgDcqfwmAg9MZ7WiU1jFyi0UOCS8wDr7zB0FK2GjROUQAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2606:4700:7::a29f:8955,2606:4700:7::a29f:8a55

;; Query time: 133 msec
;; SERVER: <LOCAL_DNS_FQDN>#53(<LOCAL_DNS_IP>) (UDP)
;; WHEN: Wed Aug 07 11:26:35 +03 2024
;; MSG SIZE  rcvd: 204

$ dig https defo.ie @<LOCAL_DNS_FQDN>

; <<>> DiG 9.18.27 <<>> https defo.ie @<LOCAL_DNS_FQDN>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39170
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;defo.ie.           IN  HTTPS

;; ANSWER SECTION:
defo.ie.        1800    IN  HTTPS   1 . ipv4hint=213.108.108.101 ech=AED+DQA8QwAgACDggLnnbB9c2cQ57n9Y9fcJ+kEN6fgBPvzCicfmx1+eJAAEAAEAAQANY292ZXIuZGVmby5pZQAA ipv6hint=2a00:c6c0:0:116:5::10

;; Query time: 276 msec
;; SERVER: <LOCAL_DNS_FQDN>#53(<LOCAL_DNS_IP>) (UDP)
;; WHEN: Wed Aug 07 11:39:02 +03 2024
;; MSG SIZE  rcvd: 149

The server is an up-to-date PiHole, which in turn uses AdGuard's dnsproxy as an upstream, which in turn uses multiple DoH servers as its upstreams. PiHole is announced network-wide via DHCP options and communicates with clients across LAN in plain.

With that configuration, Cromite as of now does not enable ECH automatically and lacks any kind of toggle in chrome://flags to enable it manually (there used to be a "#encrypted-client-hello" flag, but it is missing both from Cromite and vanilla Chrome, both up-to-date).

Firefox, on the other hand, works with my config, after I've upgraded it to v129 and switched required flags manually (by default, it still requires DoH to be enabled in browser, but v129 now can be forced to use ECH even with plain DNS servers).

Hence, this issue.

uazo commented 3 months ago

would you like to try chromium vanilla? I need to find out if one of my patches is to blame

uazo commented 3 months ago

but it is missing both from Cromite and vanilla Chrome, both up-to-date).

ah sorry

uazo commented 3 months ago

try with chrome://net-export/ the best would be with maximum information, but I realise that there might be confidential information. maybe you check first with https://netlog-viewer.appspot.com/#import

giantplaceholder commented 3 months ago

but it is missing both from Cromite and vanilla Chrome, both up-to-date).

ah sorry

Yeah, I've just updated today to 127.0.6533.99 on vanilla Chrome.

For reference, this is what I see on at https://crypto.cloudflare.com/cdn-cgi/trace when trying to use Cromite or vanilla Chrome:

fl=<censored>
h=crypto.cloudflare.com
ip=<censored>
ts=<censored>
visit_scheme=https
uag=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36
colo=<censored>
sliver=none
http=http/2
loc=<censored>
tls=TLSv1.3
sni=plaintext
warp=off
gateway=off
rbi=off
kex=X25519Kyber768Draft00

And this is the same, but with Firefox v129 (boolean flags network.dns.native_https_query, network.dns.echconfig.enabled and network.dns.http3.echconfig.enabled all set to true). Disregard that UA is set to Tor Browser, this is intentional.

fl=<censored>
h=crypto.cloudflare.com
ip=<censored>
ts=<censored>
visit_scheme=https
uag=Mozilla/5.0 (Windows NT 10.0; rv:102.0) Gecko/20100101 Firefox/102.0
colo=<censored>
sliver=none
http=http/2
loc=<censored>
tls=TLSv1.3
sni=encrypted
warp=off
gateway=off
rbi=off
kex=X25519Kyber768Draft00

And here's the small excrept from network event log (tested with defo.ie, which is definitely ECH enabled):

664: SOCKET
https://defo.ie <https://defo.ie same_site>
Start Time: 2024-08-07 13:15:05.245

t=4246 [st=   0] +SOCKET_ALIVE  [dt=2480+]
                  --> source_dependency = 662 (SSL_CONNECT_JOB)
t=4246 [st=   0]   +TCP_CONNECT  [dt=59]
                    --> address_list = [
                          "213.108.108.101:443"
                        ]
                    --> aliases = []
t=4246 [st=   0]      TCP_CONNECT_ATTEMPT  [dt=59]
                      --> address = "213.108.108.101:443"
t=4305 [st=  59]   -TCP_CONNECT
                    --> local_address = "<censored>:45236"
                    --> remote_address = "213.108.108.101:443"
t=4305 [st=  59]   +SSL_CONNECT  [dt=71]
t=4306 [st=  60]      SSL_HANDSHAKE_MESSAGE_SENT
                      --> bytes =
                        <TLS data removed>
                      --> type = 1
t=4306 [st=  60]      SOCKET_BYTES_SENT
                      --> byte_count = 2020
                      --> bytes =
                        <TLS data removed>
t=4366 [st= 120]      SOCKET_BYTES_RECEIVED
                      --> byte_count = 506
                      --> bytes =
                        <TLS data removed>
t=4366 [st= 120]      SSL_HANDSHAKE_MESSAGE_RECEIVED
                      --> bytes =
                        <TLS data removed>
                      --> type = 2
t=4367 [st= 121]      SSL_HANDSHAKE_MESSAGE_RECEIVED
                      --> bytes =
                       <handshake data removed>
                      --> type = 8
t=4367 [st= 121]      SSL_CERTIFICATES_RECEIVED
                      --> certificates =
                             <TLS certificates omitted>

t=4367 [st= 121]     +CERT_VERIFIER_REQUEST  [dt=8]
t=4367 [st= 121]        CERT_VERIFIER_REQUEST_BOUND_TO_JOB
                        --> source_dependency = 665 (CERT_VERIFIER_JOB)
t=4375 [st= 129]     -CERT_VERIFIER_REQUEST
t=4375 [st= 129]      SSL_HANDSHAKE_MESSAGE_RECEIVED
                      --> bytes =
                        <TLS data removed>
                      --> type = 20
t=4376 [st= 130]      SSL_HANDSHAKE_MESSAGE_SENT
                      --> bytes =
                        1<TLS data removed>
                      --> type = 20
t=4376 [st= 130]      SOCKET_BYTES_SENT
                      --> byte_count = 64
                      --> bytes =
                        <TLS data removed>
t=4376 [st= 130]   -SSL_CONNECT
                    --> cipher_suite = 4865
                    --> encrypted_client_hello = false
                    --> is_resumed = true
                    --> key_exchange_group = 29
                    --> next_proto = "http/1.1"
                    --> peer_signature_algorithm = 2052
                    --> version = "TLS 1.3"
t=4376 [st= 130]   +SOCKET_IN_USE  [dt=66]

uazo commented 3 months ago

And here's the small excrept from network event log

would you also put the one concerning the dns call?

giantplaceholder commented 3 months ago

And here's the small excrept from network event log

would you also put the one concerning the dns call?

Here you go:

t=4242 [st=0] +QUIC_SESSION_POOL_JOB  [dt=3]
               --> host = "defo.ie"
               --> network_anonymization_key = "https://defo.ie same_site"
               --> port = 443
               --> privacy_mode = "disabled"
               --> proxy_chain = "[direct://]"
t=4242 [st=0]    QUIC_SESSION_POOL_JOB_BOUND_TO
                [ --> source_dependency = 659 (HTTP_STREAM_JOB)]
t=4242 [st=0]   +HOST_RESOLVER_MANAGER_REQUEST  [dt=3]
                 --> allow_cached_response = true
                 --> dns_query_type = "UNSPECIFIED"
                 --> host = "https://defo.ie"
                 --> is_speculative = false
                 --> network_anonymization_key = "https://defo.ie same_site"
                 --> secure_dns_policy = 0
t=4242 [st=0]      HOST_RESOLVER_MANAGER_IPV6_REACHABILITY_CHECK
                   --> cached = true
                   --> ipv6_available = false
t=4242 [st=0]      HOST_RESOLVER_MANAGER_CREATE_JOB
t=4242 [st=0]      HOST_RESOLVER_MANAGER_JOB_ATTACH
                  [ --> source_dependency = 661 (HOST_RESOLVER_IMPL_JOB)]
t=4245 [st=3]   -HOST_RESOLVER_MANAGER_REQUEST
t=4245 [st=3] -QUIC_SESSION_POOL_JOB

t=4242 [st=0] +HOST_RESOLVER_MANAGER_JOB  [dt=3]
               --> dns_query_types = [
                     "A",
                     "AAAA",
                     "HTTPS"
                   ]
               --> host = "https://defo.ie"
               --> network_anonymization_key = "https://defo.ie same_site"
               --> secure_dns_mode = 0
              [ --> source_dependency = 660 (QUIC_SESSION_POOL_DIRECT_JOB)]
t=4242 [st=0]    HOST_RESOLVER_MANAGER_JOB_REQUEST_ATTACH
                 --> priority = "HIGHEST"
                [ --> source_dependency = 660 (QUIC_SESSION_POOL_DIRECT_JOB)]
t=4242 [st=0]    HOST_RESOLVER_MANAGER_JOB_STARTED
t=4242 [st=0]   +HOST_RESOLVER_SYSTEM_TASK  [dt=3]
t=4242 [st=0]      HOST_RESOLVER_MANAGER_ATTEMPT_STARTED
                   --> attempt_number = 1
t=4245 [st=3]   -HOST_RESOLVER_SYSTEM_TASK
                 --> address_list = [
                       "213.108.108.101:0",
                       "[2a00:c6c0:0:116:5::10]:0"
                     ]
                 --> aliases = []
t=4245 [st=3]    HOST_RESOLVER_MANAGER_ATTEMPT_FINISHED
                 --> attempt_number = 1
t=4245 [st=3] -HOST_RESOLVER_MANAGER_JOB

uazo commented 3 months ago

now would you do one with a call to a doh server? I am only interested in the section HOST_RESOLVER_MANAGER_JOB. thanks

giantplaceholder commented 3 months ago

This is with Cloudflare set up as DoH upstream. Seems that when secure_dns_mode is set to zero (as it would when "Secure DNS" is disabled and provider is not selected explicitly) it doesn't even tries to lift data from HTTP RR and proceeds with plain SNI.

t=1757 [st=  0] +HOST_RESOLVER_MANAGER_JOB  [dt=279]
                 --> dns_query_types = [
                       "A",
                       "HTTPS"
                     ]
                 --> host = "https://defo.ie"
                 --> network_anonymization_key = "https://defo.ie same_site"
                 --> secure_dns_mode = 2
                 --> source_dependency = 663 (QUIC_SESSION_POOL_DIRECT_JOB)
t=1757 [st=  0]    HOST_RESOLVER_MANAGER_JOB_REQUEST_ATTACH
                   --> priority = "HIGHEST"
                   --> source_dependency = 663 (QUIC_SESSION_POOL_DIRECT_JOB)
t=1757 [st=  0]    HOST_RESOLVER_MANAGER_JOB_STARTED
t=1757 [st=  0]   +HOST_RESOLVER_DNS_TASK  [dt=279]
                   --> secure = true
                   --> transactions_needed = [
                         {
                           "dns_query_type": "A"
                         },
                         {
                           "dns_query_type": "HTTPS"
                         }
                       ]
t=1757 [st=  0]      DNS_TRANSACTION
                     --> source_dependency = 665 (DNS_TRANSACTION)
t=1757 [st=  0]      DNS_TRANSACTION
                     --> source_dependency = 667 (DNS_TRANSACTION)
t=2036 [st=279]      HOST_RESOLVER_DNS_TASK_EXTRACTION_RESULTS
                     --> results = [
                           {
                             "domain_name": "defo.ie",
                             "endpoints": [
                               {
                                 "address": "213.108.108.101",
                                 "port": 0
                               }
                             ],
                             "hosts": [],
                             "query_type": "A",
                             "source": "dns",
                             "strings": [],
                             "timed_expiration": "13367502085755043",
                             "type": "data"
                           }
                         ]
t=2036 [st=279]      HOST_RESOLVER_DNS_TASK_EXTRACTION_RESULTS
                     --> results = [
                           {
                             "domain_name": "defo.ie",
                             "metadatas": [
                               {
                                 "metadata_value": {
                                   "ech_config_list": "AED+DQA8cQAgACDhjTYKG/y12zAhLcm/PZwrBbPFqeH8FORsbiAZDD7aWwAEAAEAAQANY292ZXIuZGVmby5pZQAA",
                                   "supported_protocol_alpns": [
                                     "http/1.1"
                                   ],
                                   "target_name": "defo.ie"
                                 },
                                 "metadata_weight": 1
                               }
                             ],
                             "query_type": "HTTPS",
                             "source": "dns",
                             "timed_expiration": "13367500285755606",
                             "type": "metadata"
                           }
                         ]
t=2036 [st=279]   -HOST_RESOLVER_DNS_TASK
                   --> results = {
                         "aliases": [
                           "defo.ie"
                         ],
                         "canonical_names": [
                           "defo.ie"
                         ],
                         "endpoint_metadatas": [
                           {
                             "endpoint_metadata_value": {
                               "ech_config_list": "AED+DQA8cQAgACDhjTYKG/y12zAhLcm/PZwrBbPFqeH8FORsbiAZDD7aWwAEAAEAAQANY292ZXIuZGVmby5pZQAA",
                               "supported_protocol_alpns": [
                                 "http/1.1"
                               ],
                               "target_name": "defo.ie"
                             },
                             "endpoint_metadata_weight": 1
                           }
                         ],
                         "expiration": "13367482409075070",
                         "host_ports": [],
                         "hostname_results": [],
                         "ip_endpoints": [
                           {
                             "endpoint_address": "213.108.108.101",
                             "endpoint_port": 0
                           }
                         ],
                         "text_records": []
                       }
t=2036 [st=279] -HOST_RESOLVER_MANAGER_JOB

uazo commented 3 months ago

in the code I found this:

doc refers to the flag kUseDnsHttpsSvcbEnableInsecure which is, however, no longer present in the code but I found in features.cc which should be v105.

I will try to check what that flag was doing and especially why it was removed... after my holidays! :)

giantplaceholder commented 3 months ago

kUseDnsHttpsSvcbEnableInsecure

It would be great if you'd take a look. Thanks!

uazo commented 2 months ago

first of all, are you talking about android? I looked at the code, the flag is no longer present but linked to [isPrivateDnsActive](https://developer.android.com/reference/android/net/LinkProperties#isPrivateDnsActive()) API.

Basically, it appears that if the system DNS is set to ‘private’, the chromium dns client does not activate the HTTPS type request (https://issues.chromium.org/issues/40219327#comment13). can you confirm this for me?

behaviour changes depending on the version of Android, which one are you using?

giantplaceholder commented 2 months ago

first of all, are you talking about android? I looked at the code, the flag is no longer present but linked to [isPrivateDnsActive](https://developer.android.com/reference/android/net/LinkProperties#isPrivateDnsActive()) API.

Basically, it appears that if the system DNS is set to ‘private’, the chromium dns client does not activate the HTTPS type request (https://issues.chromium.org/issues/40219327#comment13). can you confirm this for me?

behaviour changes depending on the version of Android, which one are you using?

I have no idea how it behaves on Android, as I do not use Chromium-based browsers on my handheld.

The behavior I described happens on desktop Linux and desktop Windows builds, taken off the local github CI.

uazo commented 2 months ago

for desktops there is a policy to be activated: https://chromeenterprise.google/policies/#AdditionalDnsQueryTypesEnabled

ref: https://issues.chromium.org/issues/40763530

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale as there has been no recent activity in response to our request for more information. Please respond so that we can proceed with this issue.

uazo commented 1 month ago

@giantplaceholder have you tried the policy if it works?

giantplaceholder commented 1 month ago

@giantplaceholder have you tried the policy if it works?

I'm not really sure how to enable it on Linux tbh. Should I pass it as a command line flag?

giantplaceholder commented 1 month ago

OK, I have found this:

https://www.chromium.org/administrators/linux-quick-start/

I've enabled the policy per the instructions above, meaning my /etc/opt/chrome/policies/managed/policies.json has the following contents:

{
        "AdditionalDnsQueryTypesEnabled": true,
        "ShowHomeButton": true
}

(the home button one is there just to have a clear visual indication if the file's in a proper location and that changes are applied)

With that said, ECH is still not working:

https://tls-ech.dev/ says that I do not use ECH
https://rutracker.org/cdn-cgi/trace reports that SNI is plaintext (can be checked with any other Cloudflare site with a free tier, just add /cdn-cgi/trace at the end of the domain root)

On the same machine, with the same local DNS server, Firefox 130 works properly, given that network.dns.native_https_query is set to true. So it is not an issue with the resolver.

uazo commented 1 month ago

can you check the installed policies with chrome://policy ? I would also need the result as https://github.com/uazo/cromite/issues/1358#issuecomment-2273100664

/etc/opt/chrome/policies/managed/policies.json

thanks for telling me, I will have to change it to /etc/opt/ cromite /policies/managed/policies.json

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale as there has been no recent activity in response to our request for more information. Please respond so that we can proceed with this issue.

github-actions[bot] commented 1 month ago

This issue has been automatically closed as sufficient information hasn't been provided on the issue for further actions to be taken. Feel free to add more information.

uazo / cromite