unbound - 1.5.10-4 reads from /var/lib/unbound/ by default?

[vollkommenheit]

This seems a behavior change from 1.5.10-3. I have configured in my openwrt's /etc/unbound/unbound.conf:

directory "/etc/unbound" chroot: "/etc/unbound" username: "unbound"

After "opkg upgrade" to 1.5.10-4, it deleted my root.hints file "/etc/unbound/named.cache". I haven't tracked down what change is responsible for this part. This is not a big deal, so long as I can get a new named.cache from elsewhere (i.e. from one of my LAN hosts).

The unbound "opkg upgrade" also changed the ownership of /etc/unbound and a bunch of files within (e.g. /etc/unbound/root.key) from "unbound" [back] to "root". This causes unbound to fail to restart, at least without troubleshooting, because now unbound, as UID "unbound", can no longer write to /etc/unbound nor /etc/unbound/root.key. IIRC, this has been happening after the last few unbound upgrades, so a change well before 1.5.10-4 is responsible for that. This is not a big deal, but ideally "opkg upgrade" unbound should not break existing unbound with customized unbound.conf like this.

Most importantly, unbound 1.5.10-4 now by default uses /var/lib/unbound/unbound.conf (which seems to be in /tmp/lib/unbound).

/etc/init.d/unbound start unbound: iterator will use built-in root hints

ps |grep unb 20012 unbound 7460 S /usr/sbin/unbound -d -c /var/lib/unbound/unbound.conf

The culprit seems to be the following changes in 1.5.10-4:

/etc/init.d/unbound script:

start_service() { unbound_prepare

procd_open_instance procd_set_param command $PROG -d -c $UNBOUND_CONFFILE procd_set_param respawn procd_close_instance }

/usr/lib/unbound/unbound.sh

UNBOUND_VARDIR=/var/lib/unbound UNBOUND_CONFFILE=$UNBOUND_VARDIR/unbound.conf

Also, the new default in /var/lib/unbound/unbound.conf theoretically makes unbound accessible to all of the subnet neighbors out of my WAN port:

interface: 0.0.0.0 interface: ::0 ... access-control: my.wan.ipv4.addr/22 allow

whereas my customized /etc/unbound/unbound.conf has the following restrictions:

interface: 0.0.0.0 interface: ::0 access-control: 0.0.0.0/0 deny access-control: ::0 deny access-control: ::1/128 allow access-control: 127.0.0.0/8 allow access-control: 192.168.0.0/16 allow # RFC 1918

Are the new behaviors in 1.5.10-4 intended?

[hnyman]

cc maintainer @EricLuehrsen

[EricLuehrsen]

Rewording previous comment. I was trying to use a less then useful tablet like device to be quick, and get you working sooner. Apparently that didn't go well.

[vollkommenheit]

I'm fine with going without the non-built-in named.cache (root.hints), and moving to chrooting unbound in /var/lib/unbound. However, a few points are unclear:

The only file that got copied from /etc/unbound to /var/lib/unbound by the unbound 1.5.10-4 upgrade appears to be "root.key". Aside from the aforementioned /etc/unbound/named.cache that got deleted, none of the following (all owned by "unbound:unbound") got copied to /var/lib/unbound:

unbound.conf unbound.forward-zone.conf unbound.local-zone.block.conf unbound.local-zone.hosts.conf unbound_control.key unbound_control.pem unbound_server.key unbound_server.pem

Should these above have been copied automatically by the unbound 1.5.10-4 install scripts? There are "unbound.check" and "unbound.conf" in /var/lib/unbound, but those apparently are installed by 1.5.10-4 itself.

What is

uci "manual_conf 1"

? Besides having no idea how that translates to the usual "uci set [blah blah]" commands in my notes on openwrt, is this something that users with existing unbound installation (and who want to continue, although from /var/lid/unbound) need to contend with, post "opkg upgrade unbound", beginning with 1.5.10-4? Prior to that, I never have to deal with "uci" for the sake of running "unbound".

Migrating from /etc/unbound to /var/lib/unbound is a big deal (there's nothing else in /var/lib other than "unbound" and "misc"), because users like myself have to be well aware of it, in order to timely update places such as /etc/sysupgrade.conf to preserve customized configs and files through pkg upgrades. Currently, I have /etc/unbound entered in /etc/sysupgrade.conf.

[EricLuehrsen]

The package net/unbound 1.5.10-4 was explicitly intended to introduce UCI to Unbound. This gives more users with less time to learn each program access to a working recursive resolver. The full extent of the content is described in README.md. For even easier access, luci-app-unbound is also provided in the LuCI repo.

Clearly people have there own configurations and there was no intention to hijack those. The UCI includes an option to disable itself. You can also check unbound.uci for a complete primer. Anyway one entry:

config unbound
    option manual_conf '1'

Summary of key problems solved in recent packages:

• Unbound DSKEY RFC5011 is hard on flash in /etc/unbound/root.key
• DNSSEC needs time; NTP need DNS; and consumer routers have no power off clock
• Many services do not need to be 'root' and should drop privileges after port-bind
• Unbound doesn't bind to new interfaces always so a hot plug event may be needed
• Unbound doesn't manage root.hints like it does root.key
• chown in /etc/ occurs after first boot in init' and quickly expands jffs partition

Summary of behaviors to manage these:

• Copy /etc/unbound to /var/lib/unbound (tmpfs) and have unbound run normally there
  • Create tmpfs working dir and files with UID 'unbound'
  • Jail (chroot) to /var/lib/unbound for security
  • Read/write with wild abandon in tmpfs and no flash harm
  • Occasionally copy root.key back to /etc/unbound (UCI option)
  • Follows standard file system hierarchy and purpose
  • Debian NetworkManager copying isc-dhclient.conf to /var/lib/NetworkManager
• UCI only, DNSSEC runs at half strength during boot (time check excluded)
  • Hot plug on NTP update switches to full strength
  • /var/lib/unbound/unbound.time is flag-file for this one time only event
  • Common with lede/packages/network/service/dnsmasq "dnssec no time check"
• Hot plug on iface to force Unbound to rebind
  • UCI generated conf binds to all interfaces
  • UCI Access allowed to only subnets with an interface
  • Common with lede/packages/network/service/dnsmasq "local service"
• named.cache or root.hints hopeful per upstream ticket Unbound ^837

Brief History:

• 1.5.8 Remove named.cache = c0e5d5e (Jun2016)
• 1.5.9 UID 'unbound' = 414eaac (Jun2016)
• 1.5.9 procd init style = 28945ea (Jun2016)
• 1.5.10 /var/lib/unbound = cb56829 (Oct2016), note already package '-3'

Bug at 1.5.10-4

• Not all of /etc/unbound is copied to /var/lib/unbound
• README.md needs this redirection instruction for manual conf files

[vollkommenheit]

It seems /var/lib/unbound/unbound.conf always gets reset to the base version that 1.5.10-4 auto-generates, regardless of "uci set unbound.manual_conf=1" or "uci set unbound.manual_conf=0". This behavior jives with what unbound.uci says:

option manual_conf '0' Boolean. Skip all this UCI nonsense. Manually edit the configuration. Make changes to /etc/unbound/unbound.conf.

So color me confused--if customized unbound.conf is supposed to remain /etc/unbound/unbound.conf, then what is the point of needing to copy everything in /etc/unbound (including /etc/unbound/unbound.conf, which gets reinitialized once copied to /var/lib/unbound by each "/etc/init.d/unbound restart") to /var/lib/unbound?

I don't pretend to understand how exactly chroot works, but how is the stated goal of jailing unbound in /var/lib/unbound/ going to work, when customized config is going to remain in /etc/unbound? What should my customized /etc/unbound/unbound.conf have, for the "directory" and "chroot" directives?

directory: "/etc/unbound" chroot: "/etc/unbound"

or

directory: "/var/lib/unbound" chroot: "/var/lib/unbound"

or nothing?

Also, everytime I update custom include files such as "unbound.local-zone.block.conf", do I need to worry about pushing a copy to both /etc/unbound and /var/lib/unbound, or one or the other?

Currently, after "uci set unbound.manual_conf=1", it seems unbound 1.5.10-4 is honoring the configs of /etc/unbound/unbound.conf, IF I start unbound manually from the CLI, even with the following in /etc/unbound/unbound.conf:

directory: "/var/lib/unbound" chroot: "/var/lib/unbound"

However, if started with /etc/init.d/unbound, it seems to only read the barebone self-generated /var/lib/unbound/unbound.conf. This brings me the question: Can a user have the benefits of BOTH customized unbound (using /etc/unbound) AND jailing in /var/lib/unbound + keeping root.key writes on tmpfs (using /var/include/unbound)?

[EricLuehrsen]

Unbound internal jail (chroot) only works on one directory, so it all has to be there. procd jails are functional but create a lot of mount point noise for the OS to deal with. The whole idea is that /etc/unbound is cloned to /var/lib/unbound and life goes on. Anyway I just tested this.

$ vi /etc/config/unbound
:  config unbound
:    option manual_conf '1'
$ /etc/init.d/unbound restart
$ more /var/lib/unbound/unbound.conf

Also, Unbound was using the settings I put there to probe using unbound-control.

You want this in your /etc/unbound/unbound.conf.

directory: "/var/lib/unbound"
chroot: "/var/lib/unbound"

When you say "bare bones," are you saying that the UCI dumps a blank configuration with "# Autogenerated ..." comment at the top. Or are you saying that somehow its pulling from the old SquashFS default from the package, and not your edit file in OverlayFS (JFFS). Your unbound.conf should be honored and copied without tampering. The only gotcha is the move to tmpfs /var/lib/unbound.

[vollkommenheit]

The auto-generated /var/lib/unbound/unbound.conf I was referring to seems to be created by /usr/lib/unbound/unbound.sh. It comes back after each "/etc/init.d/unbound restart".

/var/lib/unbound/unbound.conf generated by UCI Wed Dec 7 23:06:32 EST 2016

remote-control: control-enable: no

server: username: unbound num-threads: 1 msg-cache-slabs: 1 rrset-cache-slabs: 1 infra-cache-slabs: 1 key-cache-slabs: 1 ... edns-buffer-size: 1280

grep 1280 /usr/lib/unbound/unbound.sh

UNBOUND_N_EDNS_SIZE=1280 config_get UNBOUND_N_EDNS_SIZE "$cfg" edns_size 1280 UNBOUND_N_EDNS_SIZE=1280

On a side note, if one runs "/etc/init.d/unbound stop" more than once, back-to-back-to-back. Every instance after the first one returns:

Command failed: Not found

I don't recall how it behaved before 1.5.10-4, but I don't think it used to return something that might cause confusion like "Command failed: Not found".

OTOH, "/etc/init.d/unbound start" always returns "unbound: iterator will use built-in root hints" on consecutive runs, which would seem as if unbound is starting more than one instance. (It's not, according to syslog.)

[EricLuehrsen]

When using the command line UCI, did you commit? Did you more/less /etc/config/unbound to check?

root@OpenWrt:~# uci set unbound.manual_conf=1
root@OpenWrt:~# uci commit unbound
root@OpenWrt:~# /etc/init.d/unbound restart

[vollkommenheit]

When using the command line UCI, did you commit?

Yeah, I did at least one commit

uci get unbound.manual_conf uci set unbound.manual_conf=1 uci commit unbound uci get unbound.manual_conf

Every "/etc/init.d/unbound restart" causes /var/lib/unbound/unbound.conf to be re-autogenerated, so any custom unbound.conf doesn't survive in /var/lib/unbound/.

Did you more/less /etc/config/unbound to check?

$cat /etc/config/unbound

config unbound option dnsmsaq_gate_name '0' option dnsmasq_link_dns '0' option dnsmasq_only_local '0' option edns_size '1280' option listen_port '53' option localservice '1' option manual_conf '0' option query_minimize '0' option rebind_localhost '0' option rebind_protection '1' option recursion 'passive' option resource 'small' option root_age '28' option ttl_min '120' option unbound_control '0' option validator '0' option validator_ntp '1'

config 1 'manual_conf'

[EricLuehrsen]

I usually $ vi /etc/config/... so I didn't realize how UCI was actually handling this. Apparently an unnamed and uninstantiated section still is a section. It has a really odd syntax.

root@OpenWrt:~# uci show unbound
...
...
root@OpenWrt:~# uci get unbound.@unbound[0].manual_conf
0
root@OpenWrt:~# uci set unbound.@unbound[0].manual_conf=1
root@OpenWrt:~# uci commit unbound
root@OpenWrt:~# uci get unbound.@unbound[0].manual_conf
1
root@OpenWrt:~# 

[vollkommenheit]

So, adding "option manual_conf '1'" to /etc/config/unbound would do the trick?

[EricLuehrsen]

You need the "config unbound" section and the "option manual_conf 1" within the section.

/etc/config/unbound

config unbound
  option manual_conf '1'

Just like LEDE/dnsmasq, someday we can name and instantiate. If you have multiple isolated VLAN, then different Unbound sessions with different DNS isolating rules... someday.

[vollkommenheit]

Yup. Actually, it worked even with two conflicting entries:

option manual_conf '0' ... option manual_conf '1'

It appears I shouldn't be lazy with not enclosing the value after = with single quotes.

[EricLuehrsen]

Glad to help and sorry for the confusion. Actually it is the syntax for unnamed sections in UCI CLI that got both of us. The config unbound becomes @unbound[0]. The format is file.section.option=value and unnamed sections are file.@type[0].option=value

$ uci show unbound
...
unbound.@unbound[0].manual_conf=1
...

Anyway, you did find a bug and I have a pull request to fix it. Please look it over.

[EricLuehrsen]

@hnyman - I see you merged #3631, we can close this issue. A new issue if 1.5.10-5 causes difficulty elsewhere. Thank you.

[vollkommenheit]

Thanks.

Eeek. As I suspected, I don't see 1.5.10-5 after "opkg update". So I can't test it, because I don't have the toolchains/compiler set up to compile openwrt ipkg from source code. I do see the part of the changes that would copy everything from /etc/unbound to /var/lib/unbound.

At this point, my concerns are more around keeping unbound configs updated going forward, through future unbound upgrades and/or openwrt reboots:

1. IIRC, I see remarks in the upstream bug filed with NLlabs (concerning root.key thrashing ROM) confirming what I suspected/feared, that stuff in /tmp (tmpfs) are lost after reboots just like on desktop OS such as macOS. So does that mean I'd need to ensure my custom files get copied from /etc/unbound to /var/lib/unbound after every reboot, or is this automation provided by openwrt or unbound via UCI or otherwise?

2. I see /etc/config/unbound has "option root_age '28'" that seems to take care of copying "root.key" from /var/lib/unbound back to /etc/unbound. Well, I was hoping I could ditch /etc/unbound entirely, in favor of /var/lib/unbound. So something that would automatically and/or on-demand copy everything from /var/lib/unbound back to /etc/unbound would seem ideal for me.

Basically, because I want to reap the benefits of the UCI/tmpfs-based unbound, I need to figure out my new post-upgrade/post-reboot workflow for it, if /var/lib/unbound does need to be rebuilt after each reboot.

[EricLuehrsen]

The scripts handle it all. The intent is keep /etc/[application] the way it should be, per standards. The intent is to copy to /var/lib/[application] to handle modifications, lock out, jail and such, also standard. (Debian NetworkManager is my best example of a long standing and common user feature package that does this).

Write your files in /etc/. Know they are copied to /var/ every init.d-start. Set chroot and working dir options to /var/. The rest should just work.

If you dont username:unbound (:root) or you dont jail (no chroot:) then unbound doesnt care. You can access both /etc/ and /var/ and everything ... which is the security point. Its Unbounds single path chroot that makes this necessary. Its the messy mounting behavior that keeps procd from being an attractive alternative.

Builbot should batch up a job soon now.

[vollkommenheit]

I think this issue can be closed (I know I can close it myself). I've been running unbound 1.5.10-5 and 1.5.10-6 problem-free.