Please switch to cloudflare for fallback DNS servers

[bigon]

Hi,

In the recent years several people have shown some reservations with the fact that resolved was failing back to the Google DNS servers in some cases.

Cloudflare has started its own public DNS service a few weeks ago and for what I understand from their announce is that they grantee (at least on paper) some privacy

We began talking with browser manufacturers about what they would want from a DNS resolver. One word kept coming up: privacy. Beyond just a commitment not to use browsing data to help target ads, they wanted to make sure we would wipe all transaction logs within a week. That was an easy request. In fact, we knew we could go much further. We committed to never writing the querying IP addresses to disk and wiping all logs within 24 hours.

Shouldn't the default fallback DNS servers be switched to cloudflare ones?

The only downside (bonus so people are fixing their setup?) I'm seeing is that some ISP were/are abusing of this (previously) unallocated range for their own usage

[yuwata]

The fallback DNS servers can be changed at build time or by configuration files.

[poettering]

So yes, @yuwata is right, the fallback DNS servers can be changed at build time and by config file, hence downstream distros as well as users have a well-defined way already to change away from the google DNS servers.

That said, I figure it might make sense to switch away from the google ones upstream too, for a variety of reasons. Question though is which provider to switch to. There's at least:

1. Google (8.8.8.8)
2. Cloudflare (1.1.1.1)
3. Quad9 (9.9.9.9)

And there are more. There's a comparison here:

https://medium.com/@nykolas.z/dns-resolvers-performance-compared-cloudflare-x-google-x-quad9-x-opendns-149e803734e5

And a wikipedia page here:

https://en.wikipedia.org/wiki/Public_recursive_name_server

I am not sure what to make of this, i.e. how to pick the best upstream one.

[bigon]

IMVHO, the default should be an "unfiltered" one.

I know that some governments are forcing providers to remove some DNS names from their DNS (ie piratebay) but using Quad9, OpenDNS or others that have an editorial line about the DNS domain they are serving is going a bit far.

Edit: From your comparison, it seems that clouldflare is the fastest as well.

[johnhtodd]

(disclaimer: I work at Quad9) Quad9 does in fact have a malware/blocklist on 9.9.9.9/2620:fe::fe but we also operate a non-filtered resolver on 9.9.9.10 (and 2620:fe::10) which has no restrictions or filters. There are a variety of technical and non-technical reasons we think Quad9 is a good solution, but I won't enumerate them here other than to say that we're a not-for-profit that does only recursive DNS as a public service, and not as an adjunct to a commercial platform, which I think mirrors many of the reasons that OSS is so powerful.

[shibumi]

I think there is no best DNS server. They all have their up and down parts.. therefore I would just leave it as it is. Upstream can change it.

[Atavic]

Performance depends from user's location.

[ott]

Perhaps it makes sense to not include a list of default DNS resolvers. In the end there is no single organization that everyone will trust and we just contribute to or create additional conflict. The world is large and there is mistrust and conflict between different parts of the world. If systemd does not make a choice, it keeps out of these conflicts and remains neutral.

Consider, for example, the current default Google Public DNS. Google is subject to US laws and its business model depends – I think this is mostly undisputed by now – on personal data. The public approval of the USA varies around the world. For example, a survey of the Pew Research Center found that 15 % of people in Jordan had a favourable view of the USA while 81 % of people in Israel had favourable view in 2017 (not that the survey is scientific or accurate), just to name two extremes that are geographically close to each other. Views on privacy also differ around the world. I'm sure among those who participate in this discussion there also multiple views of Google Public DNS.

If we chose another organization, for example, Yandex.DNS, just to name a company that is subject to Russian laws, there would be similar worries, although perhaps of different groups. We could conduct similar thought experiments about Cloudflare and Quad9 and I'm sure the results would be similar.

So I think it would be best to not make a choice. It seems to be the safest decision that also matches the way the Internet has been designed. Sometimes there is not a universal solution and software has to account for the different views of people around the world.

And please don't understand me wrong here, I don't want to take a position on whether we should or should not use Google Public DNS, Cloudflare or Quad9. It don't see a purpose of that. I merely wanted to raise the more general question whether it is even possible to reach such an agreement or make such a decision that is generally accepted.

[davidstrauss]

I support moving from Google to Cloudflare as the default if we retain a default. Cloudflare's service offering has better tunneling support, retains DNSSec validation in the recursor, and introduces stronger privacy guarantees. It's also arguably faster and almost never seems to be slower (at least in data I've seen).

[shibumi]

@yuwata @poettering @davidstrauss There is another good reason why we should change to cloudflare, because of DNS over TLS-support. The Google DNS server don't support dns over tls, they only support dns over https. I thought it's a good idea to mention this, because of the newest changes to systemd-resolved.

[johnhtodd]

Quad9 has supported DNS-over-TLS since 2017, and also support DNSCrypt and is in beta for DNS-over-HTTPS. Encryption is "table stakes" for open resolvers in my opinion, but as noted above, there are regional issues which may preclude a default choice from being enforced. I would suggest that encryption and privacy policies are components that allow (during selection) an end user to differentiate between options on a presentation list, though, with more privacy-enhanced services being presented towards the top of the list if that is the model that is chosen.

[shibumi]

@johnhtodd Isn't quad9 a project from the British police?

[johnhtodd]

No. https://www.quad9.net/quad9-yourdata/

[shibumi]

@johnhtodd Sure, this is one side of the medal but on the other side you have this:

Quad9, which has been established by IBM, the Global Security Alliance (backed by the City of London Police and Center for Internet Security) and the Packet Clearing Hous

And there I trust a company more than the government (sad, but that's what we got after the Snowden leaks)

[johnhtodd]

This is off topic, and your implications are entirely incorrect, and there's not much else that should be continued here to continue that thread - we've addressed this already. DNS security and privacy are important, I agree. It is possible to imagine hypothetical and fantastic reasons that every provider that is not your own organization is doing something nefarious. If that is the case, then don't use an external provider of any sort - every single one can be accused of some unprovable misuse of data - it is never possible to prove the negative case. This thread I hope has shifted to be about what might be implemented in systemd to ensure that any options presented in the defaults (if there are even defaults) should have documented privacy policies that adhere to the most stringent (preferably GDPR in our opinion) requirements and which are open for audit, and also which enforce client encryption and other security measures (such as DNSSEC strict validation) that provide the best possible security and privacy for end users. User security/encryption, privacy standards adherence, resolver policy, and audit compliance should be the basis for inclusion in my opinion. Several public DNS services meet this bar, several do not. It seems reasonable to allow end users to make those decisions based on their own examination of the data (external links to privacy policies? a wikipedia link to comparisons?) rather than for a default to be selected without their review.

[keszybz]

So I think it would be best to not make a choice.

We rejected this idea because it would mean that explicit configuration is always required to get a working system. Right now I can always do meson build && ninja -C build && sudo ninja -C build install and it might not be ideally configured, but it should mostly work.

We could make it easy to switch by providing something like built-in groups at configuration time: @google would give the current default, @cloudflare would give the IPv4 and IPv6 cloudflare servers, etc. We could even support that during run-time, i.e. allow the named groups to be used in the DNS= config lines.

[davidstrauss]

There is another good reason why we should change to cloudflare, because of DNS over TLS-support.

This is what I meant by "Cloudflare's service offering has better tunneling support."

[ott]

@keszybz I meant that there should be no fallback DNS resolver. Either the user manually configures a resolver or it obtains one through DHCP or RDNSS. Billions of people use operating systems that also don't have a fallback DNS resolver and are happy with that. It also conforms to the Internet architecture.

I think the dynamic of this discussion shows that the discussion will lead nowhere and that we will never be able to agree on a single organization or even just a set of objective criteria to select one. So I urge everyone to consider whether this discussion is productive.

And perhaps I can say as an anecdote that I encountered people who would never trust a US company (if they understand the technical details and had the choice). They were intelligent and understand rationally that you can't lump everyone in the USA together but they were also directly or indirectly affected by wars that the USA started or participated in. I'm sure that it's the same for other countries and other parts of the world. The USA are just an example.

The world is just not ready to make such universal agreements. We should acknowledge that.

It's also not fatal that systemd-resolved would be without a fallback DNS resolver.

I feel that I'm repeating myself and that I'm contributing to an unproductive discussion. However, I wanted to clarify what I wrote and I also felt that I had to rephrase my argument so that it's understood. So please excuse the noise.

[davidstrauss]

I can think of one reason to not switch to Cloudflare DNS from the current default, and that's weirdness from certain ISPs and networks around the 1.0.0.0/8 block and 1.1.1.1 address in particular. Some networks make the incorrect assumption that those addresses are unused and either block them or handle them weirdly.

At a minimum, I would choose 1.0.0.1 for Cloudflare DNS over 1.1.1.1, as the former should exhibit marginally fewer issues with common misconfigurations.

[keszybz]

At a minimum, I would choose 1.0.0.1 for Cloudflare DNS over 1.1.1.1

I'd expect that we chose both / all, the same as the current default is 8.8.8.8 8.8.4.4 2001:4860:4860::8888 2001:4860:4860::8844.

[bigon]

TBH, I started this discussion about switching cloudflare because I understood that the discussion about setting no fallback DNS server at all was already lost and that cloudflare was a lesser evil IMHO.

From a admin POV I still don't understand why you want to have any fallback at all.

We rejected this idea because it would mean that explicit configuration is always required to get a working system.

This is why you have DHCP (or similar mechanism), adding a fallback hides the fact that something is broken in the network configuration. DNS is already a pain enough ("Everything is a fucking DNS problem", isn't it?) so IMVHO if something fails it should fail hard. My 2¢

[mbiebl]

@bigon fwiw, I've been going back and forth on this issue myself and I've decided to switch off the default fallback in Debian, i.e. the next Debian upload will have -Ddns-servers='' for basically the same reasons as yours. I have to add that Ubuntu has been switching off the fallback for quite some time already: https://salsa.debian.org/systemd-team/systemd/commit/55d514620

[shibumi]

Well, I guess this discussion is currently moving in the wrong direction. The maintainers of systemd said already that they will not remove the default DNS Servers due to stability reasons. So can we come back to the discussion about setting a default DNS server?

I would still propose cloudflare.

[mbiebl]

@shibumi what are those "stability reasons"? Personally I've been cursing on those ISPs who instead of giving me a clean error on DNS errors redirected me to a ISP specific page. The fallback mechanism is pretty close to that.

[shibumi]

@mbiebl just checkout @keszybz last reply to this thread. It's everything said already. The maintainers will not remove the fallback servers, but they are thinking about new fallback servers as alternative to the Google DNS servers. So we should get the discussion back in this direction.

[refi64]

Since this has gone way off course, here's my third-party attempt at redirecting discussion a bit.

To summarize benefits of CloudFlare:

• DNS over TLS support.
• Privacy claims ("claims" because they're most likely correct, but we can't be 100% sure).
• As far as we know CloudFlare is a mostly neutral party.

Downsides:

• Some ISPs mess with the 1.1.1.1 and 1.0.0.1 addresses.

Honestly given all this, a CloudFlare switch might be a good idea. Worst comes to worst, Google DNS could come after in the fallback list, that way there will be no worries of compatibility issues.

Should @keszybz's group idea be a separate issue?

[ott]

@davidstrauss @kirbyfan64 You could use the RIPE Atlas to measure whether 1.1.1.1 would cause problems.

[davidstrauss]

@ott I looked into this by using my company's RIPE LIR membership. 8.8.8.8 and 1.0.0.1 perform substantially better in certain regions. Italy, for example, struggles with 1.1.1.1:

To compare, here's Italy for 8.8.8.8:

1.0.0.1, the alternative Cloudflare DNS IP, has sparser data (without commissioning a larger report) but seems better overall than 1.1.1.1. This is why I'd advocate putting 1.0.0.1 first, then 1.1.1.1.

In most regions, there isn't a serious difference. I focused on Italy because the results for 1.1.1.1 were notably bad compared to other regions.

[poettering]

Let's close this as #11666 got mrged.