Swupd update are leaving files behind after an update

[otaviobp]

Describe the bug

After an update some files that should be removed are not being removed.

To Reproduce

# sudo swupd os-install --path /opt/a -B gnome-base-libs -V 32480
# sudo swupd repair --extra-files-only --path /opt/a --no-scripts # no extra file found
# sudo swupd update --path /opt/a --no-scripts
# sudo swupd repair --extra-files-only --path /opt/a --no-scripts # Multiple extra files found

Expected behavior

• No files should be left behind
• We should have tests implemented to make sure this was really fixed

Environment (please complete the following information):

• Clear Linux OS Version: 32510
• Platform: Any

Problem reported by @rossburton on clearlinux/swupd-client#1346. More information available at that issue

[castulo]

I was able to recreate this issue following the steps described in the details. After updating, if I run swupd diagnose --extra-files-only this is what I see:

$ sudo swupd diagnose --extra-files-only --path $TARGET -S $TARGET/state --no-scripts
Diagnosing version 32700
Downloading missing manifests...
Checking for extra files under /home/castulo/testfs/target/usr
 -> Extra file: /home/castulo/testfs/target/usr/share/package-licenses/ibus/COPYING.unicode
 -> Extra file: /home/castulo/testfs/target/usr/share/package-licenses/ibus/COPYING
 -> Extra file: /home/castulo/testfs/target/usr/lib64/p11-kit-trust.so
 -> Extra file: /home/castulo/testfs/target/usr/lib64/libibus-1.0.so.5.0.521
 -> Extra file: /home/castulo/testfs/target/usr/lib64/libXvMCgallium.so
 -> Extra file: /home/castulo/testfs/target/usr/lib/python3.8/site-packages/virtualenv/dirs.py
 -> Extra file: /home/castulo/testfs/target/usr/lib/python3.8/site-packages/virtualenv/__pycache__/dirs.cpython-38.pyc
 -> Extra file: /home/castulo/testfs/target/usr/lib/python3.8/site-packages/virtualenv-20.0.5-py3.8.egg-info/zip-safe
 -> Extra file: /home/castulo/testfs/target/usr/lib/python3.8/site-packages/virtualenv-20.0.5-py3.8.egg-info/top_level.txt
 -> Extra file: /home/castulo/testfs/target/usr/lib/python3.8/site-packages/virtualenv-20.0.5-py3.8.egg-info/requires.txt
 -> Extra file: /home/castulo/testfs/target/usr/lib/python3.8/site-packages/virtualenv-20.0.5-py3.8.egg-info/entry_points.txt
 -> Extra file: /home/castulo/testfs/target/usr/lib/python3.8/site-packages/virtualenv-20.0.5-py3.8.egg-info/dependency_links.txt
 -> Extra file: /home/castulo/testfs/target/usr/lib/python3.8/site-packages/virtualenv-20.0.5-py3.8.egg-info/SOURCES.txt
 -> Extra file: /home/castulo/testfs/target/usr/lib/python3.8/site-packages/virtualenv-20.0.5-py3.8.egg-info/PKG-INFO
 -> Extra file: /home/castulo/testfs/target/usr/lib/python3.8/site-packages/virtualenv-20.0.5-py3.8.egg-info/
Inspected 15 files
  15 files found which should be deleted

Use "swupd repair --picky" to correct the problems in the system

Diagnose successful
 [100%]

Now, looking at a couple of those files in the manifests....

First, checking: /usr/share/package-licenses/ibus/COPYING

I see the file in manifest 32480/Manifest.desktop-gnomelibs as: F... bf73278990a66bce9f41970193001e90a3771c49113348c8d31e24e4907afede 32480 /usr/share/package-licenses/ibus/COPYING

And then the file is no longer referenced in any other future manifest, so the file is never marked as deleted in any manifest it is just no longer there, that is why swupd is not removing it during the update but it marks it as extra file when diagnose is run.

The other file I looked at was /usr/lib/python3.8/site-packages/virtualenv/dirs.py and the exact same thing happened. The file is referenced in 32480/Manifest.python3-basic as: F... 7fac4731e0166f3ceb06be1bbcc0ae112e1818579d449c91f30c541a4a56e803 32480 /usr/lib/python3.8/site-packages/virtualenv/dirs.py`

And then it just disappears from the other manifests.

As far as I can tell swupd seems to be working correctly and this seems more like a bug in mixer.

[castulo]

@reaganlo @jwakre , I tried transferring this issue to mixer-tools but I couldn't. Can you guys take a look at this issue?

[gtkramer]

Another way to reproduce this:

• Download https://cdn.download.clearlinux.org/releases/30880/clear/clear-30880-kvm.img.xz
• Extract image and boot with kvm
• swupd bundle-add kvm-host python-extras koji
• swupd clean --all
• swupd repair --picky (comes out basically clean)
• swupd clean --all
• while swupd check-update; do swupd update; done
• swupd clean --all
• swupd repair --picky

The last command removes a bunch of files that did not get removed as a part of the update process. It should come out with pretty much the same extra files as the first diagnose. Always working with a clean cache is a bit defensive, but keeps the cache aspect out of this.

[jwakre]

These leftover files are caused by a process error where the format number was accidentally incremented and rolled back. In v32480 the format number was accidentally incremented to 30 and in v32500 the format number was rolled back to 29. When the format number was incremented, the deleted file entries were dropped from the manifest. Future versions will not re-add the deleted file entries.

Steps to reproduce:

mixer init --format 30
sudo mixer build all --increment # initial version
echo filesystem > local-bundles/os-core
sudo mixer build all --increment # version with deleted files
sudo mixer build all --increment --format 31 # increment format number, drop deleted entries
sudo mixer build all --increment --format 30 # roll-back format number, deleted file entries are still dropped

Since deleted files are expected to be dropped after a format bump, this is expected Mixer behavior.

@gtkramer That was a valid issue, but it was fixed by #684