DHCPv6 mod not kept after firmware update reboots

fgero commented 1 year ago

The current situation (before fix)

The current solution I use to keep/re-establish the DHCPv6 mod even after a reboot is unitios-utilities on-boot-script-2.x, and more precisely their udm-boot-2x_1.0.1_all.deb package.

This udm-boot-2x package creates a udm-boot systemd service (please note the name difference, as this will become important), that will automatically start all executable shell scripts it will can find in the /data/on_boot.d/ directory. In the case of DHCPv6 mod, we provide the 05-replace-odhcp6c.sh script, that you have already copied in the /data/on_boot.d/directory, following the instructions of this repo.

The problem

Unfortunately this does not work when rebooting after a Unifi OS firmware update : the package is no longer installed and the service is not created. In fact the firmware update process erases /usr and /lib among other things (but preserves most of /etc)

This issue is there to document a potential solution to fix this.

The solution (I hope it works, I'm waiting for a firmware update to test it...)

Use the ubnt-dpkg-cache facility...AND make a udm-boot package instead of udm-boot-2x because for the udm-dpkg-cache only works when the package name and the systemd service name it installs are the same...ouch

For that, the fix will provide :

a new udm-boot_1.0.1_all.deb package file (directly provided by this repo, no longer from the unifios-utilities repo), so having the exact same name as the systemd service it installs
a install_udm_boot.sh script to add the package to the ubnt-dpkg-cache facility package list, and to install the package (+ purge the previous "-2x" one if existing)

What you will need to do to activate the fix

Of course, get the latest commit. Now, in order to activate the change, YOU NEED TO RUN ONE COMMAND IN ANY CASE (whether this is your first install of DHCPv6 mod or you already have it running with current udm-boot-2x package) :

./install_udm_boot.sh

How does it work

This script will :

add the udm-boot package to the DPKG_CACHE_UBNT_PKGS env var in /etc/default/ubnt-dpkg-cache, so that it can be automatically re-installed after a firmware update reboot
uninstall (purge) any existing udm-boot-2x package, if you had installed it previously
install the newly provided udm-boot package

As the package is in the DPKG_CACHE_UBNT_PKGS list, the Unifi hooks will add it into the /persistent/dpkg/<debian_release>/ subdiredtories as soon as the package is installed by the script, including copying the deb package file to restore it at boot.

Then, after a firmware update reboot, Unifi's script /usr/lib/ubnt/hooks/system/bootup-top/50-ubnt-dpkg-restore.py is called. This Python will call the /sbin/ubnt-dpkg-restore script. This script will then check all the packages in DPKG_CACHE_UBNT_PKGS that have a file in the /persistent/dpkg/<debian_release>/action/ directory (same name as package) If this file contains the word "install" and the deb package file is present in /persistent/dpkg/<debian_release>/packages/ then it will be installed.

fgero commented 1 year ago

So first time I could test with a firmware upgrade (UDR UnifiOS 3.1.14) : doesn't work. I see traces of udm-boot in the ubnt-dpkg-restore.py, but no installation is performed at reboot. I have to investigate more, but it's not easy to test, as I need to wait for a new firmware...

In the meantime, after a firmware upgrade, you need to run again a ./install-dhcpv6-mod.sh (and even a killall udhcpc odhcp6c after) to get the IPv6 back again...

Log extract from daemon.log at reboot :


2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: [/usr/lib/ubnt/hooks/system/bootup-top/50-ubnt-dpkg-restore.py] output as following: [dump_cache] actions:
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: [dump_cache]    udm-boot: install
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: [dump_cache]    unifi-native: unpack
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: [dump_cache]    unifi-protect: remove
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: [dump_cache] packages:  udm-boot_1.0.1_all.deb
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: unifi-native_7.4.162-21057-1_arm64.deb
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Get:1 file:/persistent/dpkg/bullseye/packages ./ InRelease
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Ign:1 file:/persistent/dpkg/bullseye/packages ./ InRelease
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Get:2 file:/persistent/dpkg/bullseye/packages ./ Release
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Ign:2 file:/persistent/dpkg/bullseye/packages ./ Release
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Get:3 file:/persistent/dpkg/bullseye/packages ./ Packages
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Ign:3 file:/persistent/dpkg/bullseye/packages ./ Packages
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Get:3 file:/persistent/dpkg/bullseye/packages ./ Packages
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Ign:3 file:/persistent/dpkg/bullseye/packages ./ Packages
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Get:3 file:/persistent/dpkg/bullseye/packages ./ Packages
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Ign:3 file:/persistent/dpkg/bullseye/packages ./ Packages
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Get:3 file:/persistent/dpkg/bullseye/packages ./ Packages [907 B]
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Reading package lists...
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: purging  unifi-protect
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Reading package lists...
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Building dependency tree...
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: restoring  unifi-native
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: restoring unifi-native
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Reading package lists...
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: Building dependency tree...
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: unifi-native is already the newest version (7.4.162-21057-1).
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: 0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: restore pacakge status: enable unifi-native
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: unifi-native restore done.
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: [dump_cache] actions:
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: [dump_cache]    udm-boot: install
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: [dump_cache]    unifi-native: unpack
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: [dump_cache]    unifi-protect: remove
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: [dump_cache] packages:  udm-boot_1.0.1_all.deb
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: unifi-native_7.4.162-21057-1_arm64.deb
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: [/usr/lib/ubnt/hooks/system/bootup-top/50-ubnt-dpkg-restore.py] stderr as following: dpkg-scanpackages: info: Wrote 2 entries to output Packages file.
2023-07-18T18:17:17+02:00 UDR bootup-invoker[1652]: rm: cannot remove '/etc/init.d/unifi-led': No such file or directory
`

fgero commented 1 year ago

OK, got the explanation. The base principle of using dpkg-cache can't work, simply because in 3.x the /etc/default/ubnt-dpkg-cache file is....reset on firmware reboots. It's in /usr/share/initramfs-tools/scripts/ubnt, extract :

        log_msg "cleanup dir ${dir}"
        for f in bin sbin lib lib64 usr etc/alternatives etc/default/unifi etc/default/ubnt-dpkg-cache etc/sudoers.d etc/pam.d etc/udev etc/syslog-ng/syslog-ng.conf; do
                rm -rf ${dir}/$f
        done

I have changed the README.md to remove mention of udm-boot.

Now, after a firmware update , you need to do :

cd /data/dhcpv6-mod
./install-dhcpv6-mod.sh

Note that this is not needed for "normal" reboots, only for UnifiOS firmware updates.

fgero / dhcpv6-mod