search

brave / brave-browser

Brave browser for Android, iOS, Linux, macOS, Windows.
https://brave.com
Mozilla Public License 2.0
17.89k stars 2.34k forks source link

DNS over HTTPS #1864

Closed jumde closed 4 years ago

jumde commented 6 years ago

Test plan

See https://github.com/brave/brave-core/pull/4314

Description

DoH is gaining traction:

https://chromium.googlesource.com/chromium/src.git/+/786929ad1cfbc97962ff5672e2469460ff535f41

https://blog.nightly.mozilla.org/2018/06/01/improving-dns-privacy-in-firefox/

Consider enabling DoH for Brave.

rht commented 5 years ago

There is a flag to enable DoH on Bromite: https://github.com/bromite/bromite/blob/ac44ac738f31a336a00880bdb20a321f04f62c89/patches/BRM061_Add-a-flag-for-DNS-over-HTTPS.patch (note: GPLv3 licensed).

fmarier commented 5 years ago

Cloudflare also offers a Tor hidden service for their DoH resolver: https://blog.cloudflare.com/welcome-hidden-resolver/

fmarier commented 5 years ago

I tested the latest Bromite patch in https://github.com/fmarier/brave-core/commit/fab95d993539c48eef449164a22badc16924a595 and noticed that the DoH stack in Chrome actually falls back to the OS-provided DNS server when the DoH server fails, which was confirmed by https://www.dnsleaktest.com/. Perhaps it's doing the DNS lookups in parallel, systemd-style.

Do we want to ship this as-is or do we want to ensure that requests are only done via the DoH provider when a user configures it like that? /cc @diracdeltas

rht commented 5 years ago

IMO, in the worst case, if the DoH is doing what systemd DoT does, it still increases redundancy in the resolving method but it would still be better to know the situation: whether Chrome's DoH does the parallel lookup or not.

diracdeltas commented 5 years ago

Nice find @fmarier. Seem that it would be best to offer the user an option of whether they want to fallback to the system DNS (assuming it's falling back, not doing lookups in parallel). If it is doing lookups in parallel, we could clearly communicate it as opportunistic DOH in the UI so people are not surprised by the leaks.

cc @tomlowenthal for thoughts

tildelowengrimm commented 5 years ago

IMO, the default we should be aiming for is DoH without any local resolver use. We should ensure that there are no local leaks. But there should be an easy switch to move to a local resolver instead. Opportunistic DoH feels like the worst of both worlds.

mKoonrad commented 5 years ago

Why DoH, not DoT? DoH is better? This is my general question about encrypted DNS service.

tildelowengrimm commented 5 years ago

There's a small minority of people for whom running Tor can be a hazardous proposition, so we're not planning to default to that. DoHoT makes sense when using private windows with Tor, though.

mKoonrad commented 5 years ago

So will Brave have DoH or DoT? If yes, which version?

tildelowengrimm commented 5 years ago

My plan is to implement DoH for general browsing, and DoHoT for private windows with Tor and other backend services if someone flips the "Use Tor for background connections." switch. But that's only a plan so far — we haven't started implementing and things can change based on the technical realities.

tildelowengrimm commented 5 years ago

@fmarier How're these changes going to be affected by the NetworkService migration in Chromium (#2351)?

References:

csagan5 commented 5 years ago

I tested the latest Bromite patch in fmarier/brave-core@fab95d9 and noticed that the DoH stack in Chrome actually falls back to the OS-provided DNS server when the DoH server fails, which was confirmed by https://www.dnsleaktest.com/. Perhaps it's doing the DNS lookups in parallel, systemd-style.

Do we want to ship this as-is or do we want to ensure that requests are only done via the DoH provider when a user configures it like that? /cc @diracdeltas

I actually noticed this some time ago in the Chromium codebase and played around a bit trying to disable it or make it an option, without success; looks like somebody else upstream was aware too because there is now this patch available: https://chromium.googlesource.com/chromium/src/+/c7bed520a42ee26aa3be1da89aad737bb54c0fab

Starting from Bromite 73.0.3683.79 it will be shipped in the SECURE mode by default, as I could not easily find a way to have a flag for it (possible modes are: OFF, SECURE, AUTOMATIC).

Perhaps it's doing the DNS lookups in parallel, systemd-style.

@fmarier no, from my interpretation of the retry loops (see dns_transaction.cc) it uses the UDP/TCP attempt as a fallback when the HTTP attempt fails (this is confirmed by some source code comments there); it is however not clear how/when the HTTP attempts would be tried again. From empirical evidence I can see that switching networks is one way to trigger them anew.

All of this is thankfully no more relevant with the secure mode enabled by default.

fmarier commented 5 years ago

Thanks for the extra pointers @csagan5.

Noting that the patch landed in chromium 74.0.3705.0.

csagan5 commented 5 years ago

This is the tiny patch I wrote to enable secure mode by default (to be used after the upstream patch previously mentioned):

It will be used in Bromite until there is either a chrome://flags flag or some other configurable user option.

fmarier commented 5 years ago

This might be relevant too: https://www.chromium.org/administrators/policy-list-3#BuiltInDnsClientEnabled

bakancs commented 5 years ago

For 1st step an optiona secure DNS could be great in Brave. Why the possibility to adding secondary private DNS servers wouldn't solve the leaking problem when the primary DoH server fails?

fmarier commented 5 years ago

Chrome is planning to experiment with turning DoH in certain circumstances: https://groups.google.com/a/chromium.org/d/topic/chromium-dev/5zDcC8uQrqU/discussion

Brave-Matt commented 5 years ago

+1 from Reddit: https://www.reddit.com/r/brave_browser/comments/cj8a2x/enabling_dns_over_https/

Psyda commented 5 years ago

Comcast is lobbying against it. We should get it soon. https://www.reddit.com/r/technology/comments/dlzb8h/comcast_is_lobbying_against_encryption_that_could/

fmarier commented 5 years ago

Chrome will be running an experiment in 79 which opportunistically upgrades users to DoH if they're already using a DNS provider which supports DoH: https://blog.chromium.org/2019/09/experimenting-with-same-provider-dns.html https://www.chromium.org/developers/dns-over-https

We might be able to turn this experiment ON for all of our users.

diracdeltas commented 5 years ago

seems this may be as easy as turning on the flag at chrome://flags/#dns-over-https for our users

fmarier commented 5 years ago

Latest Bromite patches:

jlivingood commented 4 years ago

See Microsoft's announcement on DoH below. As with Google Chrome (and in contrast to Firefox) they will not change the DNS resolver (network-assigned or user-configured) but instead will opportunistically upgrade to DoH if it is detected. That detection is via a pre-configured whitelist today, which isn't particularly scalable beyond an experiment (but I'm at the IETF mtg right now and automated / standardized discovery is a discussion topic).

We will not be making any changes to which DNS server Windows was configured to use by the user or network. Today, users and admins decide what DNS server to use by picking the network they join or specifying the server directly; this milestone won’t change anything about that. Many people use ISP or public DNS content filtering to do things like block offensive websites. Silently changing the DNS servers trusted to do Windows resolutions could inadvertently bypass these controls and frustrate our users. We believe device administrators have the right to control where their DNS traffic goes.

https://techcommunity.microsoft.com/t5/Networking-Blog/Windows-will-improve-user-privacy-with-DNS-over-HTTPS/ba-p/1014229

jlivingood commented 4 years ago

It would also be nice to use the full list of resolvers in the Chrome experiment at https://www.chromium.org/developers/dns-over-https rather than the 4 resolvers noted in https://github.com/bromite/bromite/blob/master/build/patches/Add-a-flag-for-DNS-over-HTTPS.patch. (What was the basis for selecting those?)

Speaking for our DNS team here at Comcast, we have a beta resolver in the Chrome experiment w/Google at https://doh.xfinity.com/dns-query. In terms of our DNS privacy, see https://corporate.comcast.com/privacy and https://corporate.comcast.com/stories/privacy-with-comcasts-xfinity-internet-service and note that we believe we can match privacy practices with any equivalent 'trusted recursive resolver'.

jlivingood commented 4 years ago

Finally I encourage you to at least join the mailing list of the Encrypted DNS Deployment Initiative (https://www.encrypted-dns.org/) and share any deployment experiences you have - so that it can help others learn and accelerate their deployments! :-)

fmarier commented 4 years ago

It would also be nice to use the full list of resolvers in the Chrome experiment at https://www.chromium.org/developers/dns-over-https rather than the 4 resolvers noted in https://github.com/bromite/bromite/blob/master/build/patches/Add-a-flag-for-DNS-over-HTTPS.patch.

We're planning to roll out the Chrome experiment to all Brave users, so we would be using the same list as Chrome.

Finally I encourage you to at least join the mailing list of the Encrypted DNS Deployment Initiative (https://www.encrypted-dns.org/)

Thanks for sharing, I have just joined the list.

fmarier commented 4 years ago

Speaking for our DNS team here at Comcast, we have a beta resolver in the Chrome experiment w/Google at https://doh.xfinity.com/dns-query.

It appears to still be missing from Chromium: https://cs.chromium.org/chromium/src/net/dns/dns_util.cc?l=147-224&rcl=492996ab413418291fe765ebec5711ae11b1957d

I assume it will be added soon so that it's part of the experiment?

csagan5 commented 4 years ago

(What was the basis for selecting those?)

I have selected those based on their ToS and accountability; the selection was done independently and before Chromium's built-in selection was created. Some more could be added, submissions are evaluated on a case-by-case basis here: https://github.com/bromite/bromite/issues

Finally I encourage you to at least join the mailing list of the Encrypted DNS Deployment Initiative (https://www.encrypted-dns.org/)

How does this compare to the DNS Privacy Project? I assume it's unrelated and a different type of partnership.

jlivingood commented 4 years ago

How does this compare to the DNS Privacy Project? I assume it's unrelated and a different type of partnership.

This is focused on sharing lessons learned from actual deployers right now.

jlivingood commented 4 years ago

It appears to still be missing from Chromium: https://cs.chromium.org/chromium/src/net/dns/dns_util.cc?l=147-224&rcl=492996ab413418291fe765ebec5711ae11b1957d I assume it will be added soon so that it's part of the experiment?

Should be, yes. I was surprised to see that exclusion and am reached out to the Google team now.

jlivingood commented 4 years ago

I have selected those based on their ToS and accountability; the selection was done independently and before Chromium's built-in selection was created. Some more could be added, submissions are evaluated on a case-by-case basis here: https://github.com/bromite/bromite/issues

Ack & opened an issue as you requested: https://github.com/bromite/bromite/issues/421

jlivingood commented 4 years ago

It appears to still be missing from Chromium: https://cs.chromium.org/chromium/src/net/dns/dns_util.cc?l=147-224&rcl=492996ab413418291fe765ebec5711ae11b1957d I assume it will be added soon so that it's part of the experiment?

Should be, yes. I was surprised to see that exclusion and am reached out to the Google team now.

I reached out to the Google team and they reminded me what true/false on that list means. ;-) It's nothing to do with inclusion true/false but rather whether get/post is used. For us we asked them to use GET. So as of now Comcast's DoH resolver IS listed at https://cs.chromium.org/chromium/src/net/dns/dns_util.cc?l=147-224&rcl=492996ab413418291fe765ebec5711ae11b1957d

DohUpgradeEntry("Comcast", {"75.75.75.75", "75.75.76.76", "2001:558:feed::1", "2001:558:feed::2"}, {""} /* DoT hostname */, {"https://doh.xfinity.com/dns-query{?dns}", false /* use_post */}),

fmarier commented 4 years ago

Chrome's DoH rollout plan: https://docs.google.com/document/d/1FIKUmCu3_pmtChy30hFD_9H6UsyoD5_uvhbNyRaHE6s/edit

And here are the trusted recursive resolvers that Firefox currently uses (NextDNS is new): https://wiki.mozilla.org/Security/DOH-resolver-policy#Conforming_Resolvers

tunnckoCore commented 4 years ago

There's a chrome flag (chrome://flags/#dns-over-https), but it's supported in any other platform, BUT Linux. DAMN.

btlechowski commented 4 years ago

@jumde @fmarier I noticed that the test plan mentions only Windows. It this Windows only feature?

fmarier commented 4 years ago

@btlechowski It's available on Mac and Windows. I don't have a Mac to test with though, so I couldn't provide the test plan for that platform.

Dyras commented 4 years ago

@fmarier Any ideas when this is coming to Android? https://github.com/brave/browser-android-tabs/issues/672

fmarier commented 4 years ago

@Dyras We are in the process of migrating from the current Android codebase (brave/browser-android-tabs) to the unified Desktop/Android codebase (brave/brave-core). Once that migration is complete, we should be able to enable DoH on Android too.

That said, you can also enable DNS over TLS system-wide on recent versions of Android, which would make DoH somewhat unnecessary.

Dyras commented 4 years ago

@fmarier Most public WiFis (where I live anyway) block DoT while DoH works flawlessly, all the time.

Thanks for the answer, looking forward to it :)

tunnckoCore commented 4 years ago

No word on linux support.

kjozwiak commented 4 years ago

Verification PASSED on macOS 10.15.3 x64 using the following build:

Brave | 1.7.71 Chromium: 80.0.3987.149 (Official Build) dev (64-bit)
-- | --
Revision | 5f4eb224680e5d7dca88504586e9fd951840cac6-refs/branch-heads/3987_137@{#16}
OS | macOS Version 10.15.3 (Build 19D76)
Screen Shot 2020-03-24 at 1 55 48 PM

Verified passed on macOS 10.14.6 x64 (Mojave)

Brave 1.7.71 Chromium: 80.0.3987.149 (Official Build) dev (64-bit)
Revision 5f4eb224680e5d7dca88504586e9fd951840cac6-refs/branch-heads/3987_137@{#16}
OS macOS Version 10.14.6 (Build 18G3020)
Brave 1.7.82 Chromium: 80.0.3987.162 (Official Build) dev (64-bit)
Revision f2c5dd6138153bb0658091205bd1a1717f16081a-refs/branch-heads/3987@{#1034}
OS Windows 10 OS Version 1803 (Build 17134.523)
fmarier commented 4 years ago

No word on linux support.

According to this Chromium blog post, support for Android and Linux is coming soon.

tunnckoCore commented 4 years ago

@fmarier thanks :)