Configuring DNS for PiHole - discussion

[Paraphraser]

@ukkopahis - I started to look at merging (conceptually - not git merging) your revised Pi-hole doco with the stuff I came up with before when I collided with your work.

What you wrote took me down a bit of a rabbit-hole that I'd like to explore a bit further if you can spare the time.

The starting point is to make two assumptions:

1. A Raspberry Pi 192.168.203.10 which:
   • run Pi-hole in a container.
   • should use Pi-hole for its DNS with a fall-back to 8.8.8.8.
   • should discover its IP address dynamically via DHCP (assume a static binding that always returns 192.168.203.10).
2. Every other device in the network should learn DNS=192.168.203.10 via DHCP.

If I do not edit /etc/resolvconf.conf then, out-of-the-box, the Pi-hole Pi will boot, discover 192.168.203.10 for both its own IP address and the DNS host it should use, and /etc/resolv.conf will contain:

# Generated by resolvconf
nameserver 192.168.203.10

And that, of course, will work until the Pi-hole container stops running.

Let's put a pin in that for a moment and turn to your instructions in the revised Pi-hole doco. I add these lines to /etc/resolvconf.conf:

name_servers=127.0.0.1
name_servers_append=8.8.8.8
resolv_conf_local_only=NO

The result is:

# Generated by resolvconf
nameserver 127.0.0.1
nameserver 192.168.203.10
nameserver 8.8.8.8

In effect we wind up with two instances of the same RPi, one as localhost, the other learned from DHCP. I haven't tested it but I'm assuming that this will require two time-outs before 8.8.8.8 kicks in if Pi-hole stops responding.

A variation on the theme is to skip name_servers_append in favour of doing it all with name_servers. The lines-to-be-appended change to:

name_servers="127.0.0.1 8.8.8.8"
resolv_conf_local_only=NO

The result is:

# Generated by resolvconf
nameserver 127.0.0.1
nameserver 8.8.8.8
nameserver 192.168.203.10

The "duplicate" learned from DHCP has moved to the end of the list. That would seem to be a little better but it's still not perfect.

After a bit more nosing around to see if I could somehow tell resolvconf.conf to ignore DHCP, I started looking at dhcpcd.conf. One of the recommended patches for IOTstack is to append:

allowinterfaces eth*,wlan*

That stops DHCP mucking about with all the Docker virtual interfaces - the lack of that had a tendency to cause hangs on reboot. Adding this line:

nooption domain_name_servers

and reloading the dhcp client daemon before rebuilding resolv.conf via:

$ sudo service dhcpcd reload
$ sudo resolvconf -u

results in:

# Generated by resolvconf
nameserver 127.0.0.1
nameserver 8.8.8.8

which reflects the design intention correctly. Granted, once we've stopped DHCP from interfering, we can go back to:

name_servers=127.0.0.1
name_servers_append=8.8.8.8
resolv_conf_local_only=NO

and that gives the same result in /etc/resolv.conf.

My own situation is a bit different because I have another Pi running BIND9 and DHCP. BIND9 is authoritative for my local domain so I only use Pi-hole for ad-blocking. If Pi-hole is presented with an unqualified name or fully-qualified name in my local domain, it relays the query to BIND9, otherwise the query is forwarded to 8.8.8.8 and friends. In turn, BIND9 either answers authoritatively or forwards to 8.8.8.8 and friends.

My DHCP server divvies-up clients into those that need ad-blocking services and those that don't. The former are sent to Pi-hole for DNS, the latter to BIND9. As far as I'm concerned, it's really only iDevices that truly benefit from ad-blocking (which, incidentally, is why I was so interested in the WireGuard/Pi-hole hookup - I mainly use iDevices when working remotely). That said, while DHCP defaults to sending my Macs to BIND9, I do have an alternate "network location" which forces Pi-hole for DNS if a web page I want to look at turns out to be too ad-noisy. My RPis are all headless and have no need for ad-blocking so they go straight to BIND9 which they discover from DHCP.

Basically, the Pi running BIND9/DHCP and my router are the only devices with static IP addresses. Everything else discovers its IP and DNS dynamically, either as a static binding or from a dynamic pool.

I'm explaining this so you understand that I don't have huge practical experience with the nuances of resolvconf.conf and dhcpcd.conf. Until today, I wasn't really aware of the resolv.conf/dhcpcd.conf interaction.

So, what's your take on this? Do you think it's better to:

1. Use name_servers="127.0.0.1 8.8.8.8" and just ignore the "duplicate" learned from DHCP on the basis that it will never be reached (because, barring a network outage, 8.8.8.8 will always respond); or
2. Use name_servers="127.0.0.1 8.8.8.8" and nooption domain_name_servers; or
3. Stick with name_servers plus name_servers_append and include instructions for nooption domain_name_servers?

All other things being equal, I'd probably select option 2 on the basis that nooption domain_name_servers somehow falling out of /etc/dhcpcd.conf would "fail safe" and revert to option 1, whereas option 3 would see two non-functioning addresses ahead of 8.8.8.8, but that's not really a major consideration.

Thoughts?

Another thing I noticed as I was experimenting is that "Too few arguments" seems to be a Buster thing. I have yet to encounter it on any Bullseye system.

[ukkopahis]

Before I reply to your main question, let me address your concern about the timeouts:

In effect we wind up with two instances of the same RPi, one as localhost, the other learned from DHCP. I haven't tested it but I'm assuming that this will require two time-outs before 8.8.8.8 kicks in if Pi-hole stops responding.

That's a reasonable concern. With pihole stopped, running time dig google.com @localhost on the RPi will take 18s to timeout. The resolv.conf man page says its timeout is by default only 5 seconds, that is still a delay we don't want.

Luckily resolv.conf isn't used by dig, it's used by the glibc resolver, which has something clever built-in. Even though I have the resolv.conf

nameserver 127.0.0.1
nameserver 192.168.60.28
nameserver 8.8.8.8

but don't have pihole started (or nothing else listening on port 53), there is no timeout delay:

time getent hosts google.com
2a00:1450:4026:804::200e google.com

real    0m0.020s
user    0m0.007s
sys 0m0.010s

(getent is the command line tool to directly query the glibc resolver without C-coding)

Also tested adding a random nameserver 10.1.2.3 as the first entry to resolv.conf, and yes, that will cause the 5 second timeout delay, as expected.

My theory is that nameserver IPs matching to local IPs/interfaces are recognized, and timeout immediately if there is nothing locally listening on the port. Would be nice to know exactly why it works, or even some documentation about it. But (for me) it's enough that it-just-works.

[Paraphraser]

Intriguing. I think I might be heading towards a few new tests of my own...

So, would that suggest either option 1, or what I'll call option 4:

• go with "as written" but add words to the effect of "if you see the Pi's IP in the middle, ignore it, it has no effect".

I hear what you say about it not making any difference in practice. I'm now doing what I always do in these situations which is to reflect on why I posed the question in the first place ("what was the evidence in front of me and why was I concerned?") and consider whether other people In a similar situation (ie examining their own resolv.conf and seeing an apparent duplicate) will be likely to have similar concerns and, therefore, whether it is better to address it or document it?

[Paraphraser]

On further reflection, I'm leaning towards Option 4. I was thinking about a redundancy scenario where someone might have a pair of Pi-holes, with DHCP delivering IP addresses for both PI-holes for DNS. Then it would make sense to have those in between localhost and 8.8.8.8.

[ukkopahis]

Note: I've already worked on this reply for far too long. Not happy with it, but hopefully communicates get the gist of it.

Pi-hole + possibly another local DNS server

Basic sanity assumptions/requirements:

• All IOTstack docker containers and the RPI host OS all should resolve DNS queries the same way. I think this is good to avoid nasty problems and hard to debug problems. It's also mentally much more simple. I hope this will help in reducing potential bugs.
• There is a primary DNS server[^auth], as provided by the DHCP server or configured by static IP assignment in dhcpcd.conf. This DNS server may be google, your ISP, your own router, or a your custom BIND9. But still, the primary DNS is the server you choose to trust.
  • If the DHCP server defines the IOTstack RPI as DNS, the trusted DNS server has to be configured into Pi-hole.

The current Pi-hole doc falls short in describing how to set up where Pi-hole should forward queries. It should be to the primary DNS, but this isn't documented or automated. The container can be configured to do it using the Pi-hole environment variable PIHOLE_DNS_. I can't think of a straightforward way to automate this, and even if it could be I think it would be better to just document it and have the user explicitly configure this.

Lets analyze the resolv.conf as produced by my revision of Pi-hole docs (with added line-numbers):

1:  nameserver 127.0.0.1
2:  nameserver 192.168.203.10
3:  nameserver 8.8.8.8

Line 1: Forward queries to Pi-hole. Line 2: Your primary DNS, if it isn't the Pi-hole. If you change your default DHCP provided DNS server to BIND9, this line will be updated to the BIND9's IP. Thus if Pi-hole is not running/doesn't work, the RPI will use your BIND9.

• If your RPI is your primary DNS, this is your RPI's IP, it duplicates line 1, but that doesn't matter. As my previous comment points out, any entries with local IP:s[^local] in resolv.conf will not cause timeout delay problems. Thus there is no double-timeout delay problem. Line 3: Ultimate fallback, should only be used if your RPI is the primary DNS, but Pi-hole is down.

To summarize:

1. DHCP provides the primary DNS resolver address for the local LAN. We could document for:
   1. user uses the RPI as the primary DNS server (as instructed in the current Pi-hole docs).
   2. user wants another server e.g. BIND9 in your case, an ISP server or the LAN router as the primary DNS. Pi-hole is just an "extra feature" that only affects the RPI and its docker containers.
2. On the RPI, the host and all IOTstack containers resolve DNS the same way. No confusing container-specific exceptions.
   • Any IOTstack VPN client will of course use a Pi-hole running on the RPI. But should fall back to the primary DNS as configured by DHCP (BIND9 in your case).
3. DNS lookups on the Pi itself should work even if IOTstack is down.
4. As a secondary backup, if neither pi-hole nor the primary DNS responds, fall back to 8.8.8.8

Hope this clarifies why I chose to update the current Pi-hole docs to as currently: try to use the DHCP-provided DNS before falling back to 8.8.8.8.

Conclusion: manage complexity

While this is nice and provides good flexibility, I'm starting to doubt something this complicated even belongs to IOTstack. IOTstack should provide a basic simple, it-just-works way a beginner gets everything up and running. Bumping into advanced topics about multiple DNS servers, DNS forwarding, recursive resolving, authoritative servers, reverse queries is just confusing when trying to just get a basic IOT-setup to work. A new user may believe it's something required.

Using nooption domain_name_servers certainly makes things more "elegant" for certain use-cases, but isn't required in order to fix anything. And may break some setups (like mine).

All this and other advanced stuff could be added to the docs, but separated to a new "Further reading"-subfolder. Just to be clearly separated from the supported it-just-works side of things.

add words to the effect of "if you see the Pi's IP in the middle, ignore it, it has no effect".

Good idea.

[^auth]: I use the term "primary DNS" and not "authoritative DNS", as the term "authoritative" has a special meaning when talking about DNS. [^local]: as reported by $ ip address | grep inet

[ukkopahis]

If anyone plans to write an advanced setup guide to dns, consider also mentioning:

• resolv.conf supports at most 3 nameserver entries
• resolv.conf entries are NOT used in a round-robin fashion, queries are always primarily sent to the first entry. Additional nameserver entries are used only if the previous entry is unavailable/timeouts