pop-os / upgrade

Utility for upgrading Pop!_OS and its recovery partition to new releases.
GNU General Public License v3.0
95 stars 29 forks source link

Fresh 20.04 to 22.04 upgrade fails #319

Closed jacobgkau closed 1 year ago

jacobgkau commented 1 year ago

Steps to reproduce:

The system has the upgrade complete/rebooting message much quicker than expected, and the system reboots back into version 20.04, but with all sources pointing to Jammy.

Full offline upgrader log: upgrade.log

The following output is near the top of the log file:

Output: ``` The following packages will be marked as automatically installed: libgcrypt20 lsb-release gnupg apt libc6 libstdc++6 dpkg ca-certificates libdbus-1-3 libexpat1 libglib2.0-0 libuuid1 dbus gir1.2-glib-2.0 libcrypt1 libsystemd0 libgcc-s1 kmod lsb-base libncursesw6 libtinfo6 python3 python3-yaml libudev1 init-system-helpers zlib1g util-linux base-files pop-upgrade linux-system76 sessioninstaller coreutils dmsetup dosfstools e2fsprogs locales mount libcryptsetup12 libgnutls30 python3-gi libsqlite3-0 shared-mime-info liblzma5 adduser bash systemd-sysv libpam-modules libpam-runtime procps ucf debconf libaudit1 libpam0g libselinux1 libpop-upgrade-gtk libxml2 libcom-err2 libp11-kit0 libgirepository-1.0-1 systemd kernelstub libelf1 libcap2 libseccomp2 linux-generic bzip2 libslang2 python3-dbus libzstd1 libfribidi0 xdg-user-dirs openssl libreadline8 udev libnss-systemd libpam-systemd liblz4-1 libacl1 libbz2-1.0 libblkid1 libfdisk1 libapparmor1 libgpg-error0 libkmod2 libmount1 systemd-timesyncd libip4tc2 networkd-dispatcher sudo iproute2 libdevmapper1.02.1 file libpython3-stdlib passwd liblocale-gettext-perl libext2fs2 libpcre3 cron debianutils rsyslog mime-support gpgv libapt-pkg6.0 ubuntu-keyring apt-utils sensible-utils libdb5.3 libattr1 mawk libcap-ng0 xz-utils isc-dhcp-client iputils-ping libpopt0 netbase libidn2-0 libbsd0 libsmartcols1 bsdutils less eject fdisk libnewt0.52 logrotate libncurses6 libcap2-bin tzdata netplan.io python3-netifaces software-properties-common libmnl0 console-setup console-setup-linux xkb-data keyboard-configuration kbd libgmp10 wamerican perl-base whiptail debconf-i18n libtext-iconv-perl libtext-wrapi18n-perl libtext-charwidth-perl libyaml-0-2 distro-info-data libsodium23 tar python3-pkg-resources libss2 logsave poppler-data libmagic1 sed libunistring2 libpcre2-8-0 hunspell-ar hunspell-de-at-frami hunspell-de-ch-frami hunspell-de-de-frami hunspell-en-au hunspell-en-ca hunspell-en-gb hunspell-en-us hunspell-en-za hunspell-es hunspell-it hunspell-pt-br hunspell-pt-pt hunspell-ru libxtables12 libatm1 libisc-export1105 isc-dhcp-common readline-common libargon2-1 libaudit-common libc-bin python3.8 libpam-cap libestr0 libfastjson4 libglib2.0-data libtasn1-6 libmagic-mgc libpython3.8-stdlib libnetplan0 libpam-modules-bin libprocps8 libsemanage-common netcat-openbsd login python3-six xxd vim-common python3-minimal python3-cffi-backend python3-nacl python3-pymacaroons ubuntu-advantage-tools vim-tiny wbrazilian wbritish wfrench witalian wngerman wogerman wportuguese wspanish wswiss gcc-10-base libffi7 libssl1.1 libjson-c4 libicu66 libhogweed5 libnettle7 libdns-export1109 libmpdec2 libpython3.8-minimal libsemanage1 libsepol1 python3.8-minimal Do you want to continue? [y/N] Y Reading package lists... Building dependency tree... Reading state information... The following additional packages will be installed: fakeroot gcc-12-base gnome-keyring libc-bin libc-dev-bin libc6-dev libegl-mesa0 libfakeroot libffi8 libgbm1 libgcrypt20 libgl1-mesa-dri libglapi-mesa libglib2.0-bin libglib2.0-dev libglib2.0-dev-bin libglx-mesa0 libgmp10 libgnutls30 libhogweed6 libllvm14 libmount-dev libnettle8 libnsl-dev libnsl2 libnss-nis libnss-nisplus libselinux1 libselinux1-dev libsepol-dev libsepol2 libssl3 libstdc++6 libtirpc-common libtirpc-dev libtirpc3 libwayland-client0 libx11-6 libx11-xcb1 locales network-manager-pptp network-manager-pptp-gnome rpcsvc-proto zlib1g-dev Suggested packages: glibc-doc rng-tools libgirepository1.0-dev libglib2.0-doc libxml2-utils gnutls-bin libteam-utils Recommended packages: libc-devtools The following packages will be REMOVED: libsepol1-dev E: Internal error, InstallPackages was called with broken packages! ```

The E: Internal error, InstallPackages was called with broken packages! error then repeats after later steps in the log file.

On my test VM, after I booted back into 20.04, the Jammy sources were valid. Performing a regular apt upgrade (as many users are unaware of full-upgrade), I noticed that I was asked whether or not to upgrade various files in /etc, including our /etc/os-release and /etc/lsb-release files, with the default choice being "no":

image

This explains why some users have appeared on Mattermost with mostly-upgraded systems but with those files still reporting that they're on the prior release.

After finishing the apt upgrade, there were still a number of packages that needed to be upgraded using apt full-upgrade. Doing that and then rebooting, I dropped into a busybox prompt. Not sure of the actual end state of the system, but we should probably focus on the earlier pop-upgrade failure first (as everything after that is dependent on how the user tries to proceed.)

jacobgkau commented 1 year ago

I attempted to replicate this again today, running upgrade.sh manually. The script did not appear to fail. After it finished, /etc/os-release still said 20.04. After rebooting, I am in 22.04, and all packages are up-to-date, but pop-upgrade still offers to "upgrade" to 22.04 since /etc/os-release is wrong.

image

This feels like something is wrong with the pop-default-settings packaging causing those files not to be updated. However, a possible workaround might simply be force reinstalling the /etc/os-release/etc/lsb-release files as described here at the end of the upgrade.sh script.

jacobgkau commented 1 year ago

https://github.com/pop-os/default-settings/pull/163 touched the conffiles configuration of pop-default-settings. I'm beginning to suspect this issue was caused by that PR.

mmstick commented 1 year ago

Perhaps it'll be fixed by renaming it back to conffiles from pop-default-settings.conffiles.

jacobgkau commented 1 year ago

I see in the upgrade.sh output where pop-upgrade is leaving the old pop-default-settings conf files by default:

image

Perhaps it'll be fixed by renaming it back to conffiles from pop-default-settings.conffiles.

I think the breaking change in https://github.com/pop-os/default-settings/commit/7fcad352e1fc8733ec0c4c9c787fc737783aad96 might have been that we used to override dh_installdeb and copy our conffiles file over manually after the automatic version has been run. Without doing that, dh_installdeb is going to include any file under /etc by default, and we leave it there (by not copying over it.)

So maybe the naming doesn't matter so much as that override. (The new naming is correct if we want to have it use that and then add its auto-generated ones; doesn't matter so much if we use the "correct" name or not if we're overriding?)

jacobgkau commented 1 year ago

I'm working on this in a branch: https://github.com/pop-os/default-settings/tree/conffiles_jammy

jacobgkau commented 1 year ago

image

/etc/os-release is displaying correctly with the fix in https://github.com/pop-os/default-settings/pull/165.