NLnetLabs / unbound

Unbound is a validating, recursive, and caching DNS resolver.
https://nlnetlabs.nl/unbound
BSD 3-Clause "New" or "Revised" License
3.15k stars 359 forks source link

Systemctl degradation: unbound-resolvconf.service #1161

Closed melroy89 closed 2 weeks ago

melroy89 commented 4 weeks ago

Describe the bug

I'm running Unbound successfully now, however I still notice a systemctl degradation issue with a specific unbound-resolveconf service? Due to loopback device failure by the unbound-helper script.

This is not what you want to see happening in production.

unbound-resolvconf.service loaded    failed     failed    Unbound asyncronous resolvconf update helper

To reproduce Steps to reproduce the behavior:

  1. Start Unbound under Ubuntu Server 24.04
  2. Run: sudo systemctl status unbound-resolvconf.service

Output:

Oct 24 21:47:05 ubuntu-server systemd[1]: Started unbound-resolvconf.service - Unbound asyncronous resolvconf update helper.
Oct 24 21:47:05 ubuntu-server resolvconf[2546758]: Dropped protocol specifier '.unbound' from 'lo.unbound'. Using 'lo' (ifindex=1).
Oct 24 21:47:05 ubuntu-server resolvconf[2546758]: Failed to set DNS configuration: Link lo is loopback device.
Oct 24 21:47:05 ubuntu-server systemd[1]: unbound-resolvconf.service: Main process exited, code=exited, status=1/FAILURE
Oct 24 21:47:05 ubuntu-server systemd[1]: unbound-resolvconf.service: Failed with result 'exit-code'.

Expected behavior

No errors. No failures.

System:

Version 1.19.2

Configure line: --build=x86_64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-option-checking --disable-silent-rules --libdir=${prefix}/lib/x86_64-linux-gnu --runstatedir=/run --disable-maintainer-mode --disable-dependency-tracking --with-pythonmodule --with-pyunbound --enable-subnet --enable-dnstap --enable-systemd --enable-cachedb --with-libhiredis --with-libnghttp2 --with-chroot-dir= --with-dnstap-socket-path=/run/dnstap.sock --disable-rpath --with-pidfile=/run/unbound.pid --with-libevent --enable-tfo-client --with-rootkey-file=/usr/share/dns/root.key --enable-tfo-server
Linked libs: libevent 2.1.12-stable (it uses epoll), OpenSSL 3.0.13 30 Jan 2024
Linked modules: dns64 python cachedb subnetcache respip validator iterator
TCP Fastopen feature available

Additional information

unbound-resolvconf seems to be enabled by default and also on startup.

I could NOT even find any information about this service in the documentation: https://unbound.docs.nlnetlabs.nl/en/latest/ (nothing mentioned about resolveconf at all 😖 )

Here is the content of /usr/lib/systemd/system/unbound-resolvconf.service:

# This is a helper unit to register local unbound resolver daemon in
# resolv.conf.  This is done asyncronously in order not to slow down
# the main unbound service startup, since this will trigger resolvconf
# reload of other services which might be a lengthly procedure.
# See #826241 for the origins of this service.
#
# Can this whole thing be done using ExecStartPost= in unbound.service?

[Unit]
Description=Unbound asyncronous resolvconf update helper
After=unbound.service
PartOf=unbound.service
ConditionFileIsExecutable=/sbin/resolvconf

[Service]
RemainAfterExit=yes
ExecStart=/usr/libexec/unbound-helper resolvconf_start
ExecStop=/usr/libexec/unbound-helper resolvconf_stop

[Install]
WantedBy=unbound.service
wcawijngaards commented 4 weeks ago

We do not make the unbound-helper service. That explains why it is not in our documentation. I have no idea how it works.

melroy89 commented 4 weeks ago

OK, I try to read a bit into this service. Under Debian/Ubuntu it seems to be related to the resolvconf command, that register a DNS server (admin domain configuration) with systemd-resolved.

I found a section on the Pi-Hole documentation. Describing that this unbound-resolvconf.service is adding the Unbound service to the /etc/resolv.conf file, in this case adding: nameserver 127.0.0.1.

In my case, this is actually correct. Unbound is running on port 53 and listening on localhost. I already created a file at (as you know I recently documented this to the Unbound documentation): /etc/systemd/resolved.conf.d/unbound-changes.conf with the content:

[Resolve]
DNS=127.0.0.1
DNSSEC=yes
DNSStubListener=no

Meaning it should also disable the systemd-resolved service listening on port 53.

Next, I took a peek into this unbound-helper shell script (yes, it's just a shell script), which is used by this unbound-resolvconf.service. Allowed input parameters with this script are: resolvconf_start|resolvconf_stop|chroot_setup|chroot_teardown|root_trust_anchor_update.

Since resolveconf_start is causing the issue here. I will focus on that one for now:

UNBOUND_CONF="/etc/unbound/unbound.conf"
UNBOUND_BASE_DIR="${UNBOUND_CONF%/*}"
CHROOT_DIR="$(unbound-checkconf -o chroot)"

DNS_ROOT_KEY_FILE="/usr/share/dns/root.key"
ROOT_TRUST_ANCHOR_FILE="/var/lib/unbound/root.key"

# Override these variables by editing or creating /etc/default/unbound.
RESOLVCONF=true
ROOT_TRUST_ANCHOR_UPDATE=true

[....]

do_resolvconf_start() {
    [ false != "$RESOLVCONF" -a -x /sbin/resolvconf ] || return 0

    unbound-checkconf $CHROOT_DIR/$UNBOUND_CONF -o interface | {
        default=yes
        while read interface; do
            default=
            # XXXX here, only localhost and all-zero addresses are handled
            # in case some other IP is specified it will not work
            case "$interface" in
              ( 0.0.0.0 | 127.0.0.1 ) echo "nameserver 127.0.0.1" ;;
              ( ::0 | ::1 ) echo "nameserver ::1" ;;
            esac
        done
        [ -z "$default" ] ||
            # unbound defaults to listening on localhost
            echo "nameserver 127.0.0.1"
    } | /sbin/resolvconf -a lo.unbound
}

So it's basically executing echo "nameserver 127.0.0.1" | /sbin/resolvconf -a lo.unbound in my case (-a flag will register per-interface DNS server). Running the command manually result indeed in the exact same error message:

Dropped protocol specifier '.unbound' from 'lo.unbound'. Using 'lo' (ifindex=1).
Failed to set DNS configuration: Link lo is loopback device.

What is resolvconf exactly? I dunno, see the docs: https://www.freedesktop.org/software/systemd/man/latest/resolvectl.html. So there seems to be an interface between systemd-resolved and resolvconf possible.

Despite Unbound didn't wrote this above script or unbound-resolvconf service, it seems to be widely spread in both Debian and Ubuntu systems.

The Pi-hole documentation stated that you can disabled this service fully. And I believe this is correct for me as well. Just wondering if this service will ever work on the lo.unbound interface!? Or what the intent was with this lo.unbound interface name..

Maybe still a good idea to document something about this service... Regardless. It's still part of Ubuntu Server 24.04 LTS after all.

wcawijngaards commented 4 weeks ago

I think resolvconf is a method to access the /etc/resolv.conf file. If your edit already works, the program seems unneeded. From the code snippet and the failure it looks like the lo.unbound name both causes the failure but also the name is fixed in the script, not sure how that would work. If it worked, it would likely do that change that was made by adding the unbound-changes.conf but by the helper script, which is nice. Since you already have the added file in resolved.conf.d, I also believe it is not needed for this set up.

melroy89 commented 4 weeks ago

I believe you're right.. I don't need it, and it only causes errors.

Disable the service:

sudo systemctl disable --now unbound-resolvconf.service

And/or mask it:

sudo systemctl mash unbound-resolvconf.service

Should we add it to the docs?

wcawijngaards commented 4 weeks ago

That would be neat, to describe how to turn off the failures. The description perhaps like, 'If in systemctl the unbound-resolvconf.service shows trouble, that is doing the same thing as the added /etc/systemd/resolved.conf.d/unbound-changes.conf file, the failures can be removed by disabling the service, with ..commandline..' ?

melroy89 commented 4 weeks ago

PR created: https://github.com/NLnetLabs/unbound-manual/pull/55