Silverblue F40: Upgrade/downgrade ping-pong with 4 packages

[jmaibaum]

Describe the bug On Fedora Silverblue 40, since a couple of days, 4 packages get upgraded with every rpm-ostree upgrade, and subsequently downgraded again with every other rpm-ostree upgrade. In other words, these packages ping-pong between 2 specific versions with every upgrade:

First these get upgraded:

Upgraded:
  binutils 2.41-34.fc40 -> 2.41-35.fc40
  binutils-gold 2.41-34.fc40 -> 2.41-35.fc40
  perl-URI 5.27-1.fc40 -> 5.28-1.fc40
  zstd 1.5.5-5.fc40 -> 1.5.6-1.fc40

And after the next rpm-ostree upgrade the same packages get downgraded again:

Downgraded:
  binutils 2.41-35.fc40 -> 2.41-34.fc40
  binutils-gold 2.41-35.fc40 -> 2.41-34.fc40
  perl-URI 5.28-1.fc40 -> 5.27-1.fc40
  zstd 1.5.6-1.fc40 -> 1.5.5-5.fc40

In the list of my available deployments all entries start from the same base commit (after doing 3 rpm-ostree upgrades in a row, and rebooting after the first 2):

$ rpm-ostree status
State: idle
Deployments:
  fedora:fedora/40/x86_64/silverblue
                  Version: 40.20240414.n.0 (2024-04-14T08:08:47Z)
               BaseCommit: 378ff9bfb29d4b43d24b86402155caa014d9f21bfb5e1c73d890f1c2aa8c0f89
             GPGSignature: Valid signature by 115DF9AEF857853EE8445D0A0727707EA15B79CC
                     Diff: 4 downgraded
          LayeredPackages: containernetworking-plugins flatpak-builder git-email
                           gstreamer-plugins-espeak gstreamer1-plugin-openh264 hut openssl

● fedora:fedora/40/x86_64/silverblue
                  Version: 40.20240414.n.0 (2024-04-14T08:08:47Z)
               BaseCommit: 378ff9bfb29d4b43d24b86402155caa014d9f21bfb5e1c73d890f1c2aa8c0f89
             GPGSignature: Valid signature by 115DF9AEF857853EE8445D0A0727707EA15B79CC
          LayeredPackages: containernetworking-plugins flatpak-builder git-email
                           gstreamer-plugins-espeak gstreamer1-plugin-openh264 hut openssl

  fedora:fedora/40/x86_64/silverblue
                  Version: 40.20240414.n.0 (2024-04-14T08:08:47Z)
               BaseCommit: 378ff9bfb29d4b43d24b86402155caa014d9f21bfb5e1c73d890f1c2aa8c0f89
             GPGSignature: Valid signature by 115DF9AEF857853EE8445D0A0727707EA15B79CC
          LayeredPackages: containernetworking-plugins flatpak-builder git-email
                           gstreamer-plugins-espeak gstreamer1-plugin-openh264 hut openssl

To Reproduce

1. rpm-ostree upgrade (4 packages receive upgrade)
2. reboot
3. rpm-ostree upgrade (4 packages receive downgrade)
4. reboot
5. start over

Expected behavior The deployment stays stable after step 2. i.e. no further up-/downgrade with the exact same base commits.

OS version:

State: idle
BootedDeployment:
● fedora:fedora/40/x86_64/silverblue
                  Version: 40.20240414.n.0 (2024-04-14T08:08:47Z)
               BaseCommit: 378ff9bfb29d4b43d24b86402155caa014d9f21bfb5e1c73d890f1c2aa8c0f89
             GPGSignature: Valid signature by 115DF9AEF857853EE8445D0A0727707EA15B79CC
          LayeredPackages: containernetworking-plugins flatpak-builder git-email gstreamer-plugins-espeak gstreamer1-plugin-openh264 hut openssl

[francoism90]

I don't know if this fixes your issue, but I also had this issue in the past.

In most cases I fully reset the rpm-ostree, and reinstall everything to get it working again:

rpm-ostree reset
systemctl reboot

After rebooting:

rpm-ostree upgrade
systemctl reboot

Check you are in a clean state again:

rpm-ostree status

Afterwards reinstall your packages again.

[jmaibaum]

This seems to have done the trick. Thanks. I will keep an eye on it and report back in a couple of days. Yet, I wonder what caused this in the first place?

[francoism90]

@jmaibaum This may happen when an overlay package is either part of the image or has deps issues.

You could debug this by removing each overlayed package and try again to update. But in most cases a reset is much easier.

[jmaibaum]

Ok, well, could be that this happened with one of the packages after the rebase from f39 to f40 I guess. Though I rebased a couple weeks ago already but this issue only appeared some time last week.

[travier]

Closing as there is a workaround and this should potentially be reported upstream in rpm-ostree to get it fixed.

[jmaibaum]

Since the issue has gone since I applied the workaround, I am happy for now. In case it appears again, I will report it to rpm-ostree upstream. Thanks.