[Bug] - dnf upgrade fails with kernel-livepatch packages installed

[nosnilmot]

Describe the bug

With Kernel Live Patching enabled, the default configuration can get in a state where it is impossible to simply dnf upgrade.

To Reproduce

1. Install Amazon Linux 2023
2. Enable Kernel Live Patching
3. Enable Automatic updates (systemctl enable --now dnf-automatic-install.timer)
4. Wait (a while - months)
5. Realize Amazon Linux 2023 does NOT get updates by default
6. Panic
7. Switch to latest releasever (echo latest | sudo tee /etc/dnf/vars/releasever)
8. Try to update
9. Panic again

Expected behavior dnf should install available updates

Screenshots

$ sudo dnf upgrade
Last metadata expiration check: 9:19:15 ago on Wed Jul  3 20:55:56 2024.
Error: 
 Problem: The operation would result in removing the following protected packages: kernel
(try to add '--skip-broken' to skip uninstallable packages)

$ sudo dnf -d 10 upgrade
timer: config: 1 ms
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, kpatch, needs-restarting, playground, release-notification, repoclosure, repodiff, repograph, repomanage, reposync, supportinfo
DNF version: 4.14.0
Command: dnf -d 10 upgrade 
Installroot: /
Releasever: latest
cachedir: /var/cache/dnf
Base command: upgrade
Extra commands: ['-d', '10', 'upgrade']
User-Agent: constructed: 'libdnf (Amazon Linux 2023; generic; Linux.aarch64)'
repo: using cache for: amazonlinux
amazonlinux: using metadata from Thu Jun 27 00:00:00 2024.
repo: using cache for: kernel-livepatch
kernel-livepatch: using metadata from Mon Jun 24 00:00:00 2024.
Last metadata expiration check: 9:57:13 ago on Wed Jul  3 20:55:56 2024.
timer: sack setup: 145 ms
--> Starting dependency resolution
--> Finished dependency resolution
timer: depsolve: 143 ms

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/dnf/cli/main.py", line 130, in cli_run
    ret = resolving(cli, base)
  File "/usr/lib/python3.9/site-packages/dnf/cli/main.py", line 166, in resolving
    base.resolve(cli.demands.allow_erasing)
  File "/usr/lib/python3.9/site-packages/dnf/base.py", line 929, in resolve
    raise exc
dnf.exceptions.DepsolveError: 
 Problem: The operation would result in removing the following protected packages: kernel
Error: 
 Problem: The operation would result in removing the following protected packages: kernel
(try to add '--skip-broken' to skip uninstallable packages)
Cleaning up.

Additional context

It is possible to get most updates installed by excluding kernel and kernel-livepatch packages:

$ sudo dnf upgrade --exclude kernel --exclude kernel-livepatch\*
Last metadata expiration check: 9:25:31 ago on Wed Jul  3 20:55:56 2024.
Dependencies resolved.
====

....

Complete!

Significantly increasing installonly_limit in /etc/dnf/dnf.conf (to >= number of installed kernel-livepatch, 14, although that may not be the actual constraint) also allows dependency resolution to complete:

sudo dnf upgrade
Last metadata expiration check: 10:06:43 ago on Wed Jul  3 20:55:56 2024.
Dependencies resolved.
==========================================================================================
 Package                         Arch     Version                 Repository         Size
==========================================================================================
Installing:
 kernel                          aarch64  6.1.34-56.100.amzn2023  amazonlinux        27 M
 kernel                          aarch64  6.1.34-58.102.amzn2023  amazonlinux        27 M
 kernel                          aarch64  6.1.34-59.116.amzn2023  amazonlinux        27 M
 kernel                          aarch64  6.1.38-59.109.amzn2023  amazonlinux        27 M
 kernel                          aarch64  6.1.49-69.116.amzn2023  amazonlinux        27 M
 kernel                          aarch64  6.1.49-70.116.amzn2023  amazonlinux        27 M
 kernel                          aarch64  6.1.52-71.125.amzn2023  amazonlinux        27 M
Upgrading:
 kernel-livepatch-6.1.29-50.88   aarch64  1.0-2.amzn2023          kernel-livepatch   23 k
 kernel-livepatch-6.1.34-56.100  aarch64  1.0-1.amzn2023          kernel-livepatch   15 k
 kernel-livepatch-6.1.34-58.102  aarch64  1.0-1.amzn2023          kernel-livepatch   15 k
 kernel-livepatch-6.1.34-59.116  aarch64  1.0-1.amzn2023          kernel-livepatch   15 k
 kernel-livepatch-6.1.38-59.109  aarch64  1.0-1.amzn2023          kernel-livepatch   13 k
 kernel-livepatch-6.1.49-69.116  aarch64  1.0-2.amzn2023          kernel-livepatch   51 k
 kernel-livepatch-6.1.49-70.116  aarch64  1.0-2.amzn2023          kernel-livepatch   51 k
 kernel-livepatch-6.1.52-71.125  aarch64  1.0-2.amzn2023          kernel-livepatch   51 k
 kernel-livepatch-6.1.55-75.123  aarch64  1.0-1.amzn2023          kernel-livepatch   27 k

Transaction Summary
==========================================================================================
Install  7 Packages
Upgrade  9 Packages

Total download size: 189 M
Is this ok [y/N]:

[zcobol]

There's a discussion about AL2023 automatic update strategies at https://github.com/amazonlinux/amazon-linux-2023/discussions/395

[andreasstieger]

Getting someone to look at this for triage/action.