dist-upgrade with --allow-vendor-change does not move all packages to higher priority repositories

[mpeter50]

On my openSUSE Leap 15.3 installation I was trying to update KDE software to more up to date versions. For instructions I was following this page: https://en.opensuse.org/SDB:KDE_repositories Specifically I ran the commands listed in the "Example: Latest KDE software releases on Leap:" section, since those seemed to fit my case.

At first it seemed the dist-upgrade was completed successfully (the summary at the end has shown that a few hundred packages were updated, and the 3 other outcome categories had a count of 0), but it turns out there are some packages that were left with the older version, that come from the official repos. First I noticed that the System Settings app has lost quite a few menu items from the left. From it's about menu, it turns out it wasn't updated along with other KDE packages, and the menu items are missing because of incompatibility between the versions. Running zypper up tells me about a few more packages that could be updated, but also that it won't do that:

It looks like these 5 packages were left out from being updated in the dist-upgrade process.

Since then I did some experimenting, and I've found a few other things that might be worth noted:

• zypper list-updates does not show these updates unless I use the --all argument. --allow-vendor-change has no effect here.

• zypper dup --from 'KDE-Frameworks' --dry-run shows that there are package conflicts. There may be other conflicts too. I used --dry-run so it might be easier to investigate the bug.

• There is already a similar issue in this repo, #290 . There it was also suggested to use the --from argument.

Why didn't Zypper tell me on the first dup run that there are package conflicts? What is the difference between --from and --allow-vendor-changes? Why did the latter skip some packages, while updating most of it?

[mlandres]

@mpeter50 If zypper dup is used for an update, there should be a solver-testcase at /var/log/updateTestcase-YYYY-MM-DD-hh-mm-ss (date and time of the update). Please pack the testcase directory together with the /var/log/zypper.log and save it somewhere in case we need it.

  cd /tmp
  tar cvjf zypper-testcase.bz2 /var/log/updateTestcase-YYYY-MM-DD-hh-mm-ss /var/log/zypper.log

[mlandres]

Most probably there was no 'vendor conflict' during the first dup.

Look at the plasma5-session package for example:

installed: 5.18.6 - bp153.4.2.3 vendor: openSUSE in a repo: 5.24.2 - lp153.639.2 vendor: obs://build.opensuse.org/KDE

Per default vendor protection is on. This means the packages from vendor openSUSE are updated by those from vendor openSUSE, no matter if another vendor distributes a package with the same name.

In general you can't tell whether a different vendor provides compatible packages regarding content or version schema. So this is the safe default.

This is why zypper lu tells that there are packages with a higher version number, but they won't be installed per default because they violate e.g. the vendor policy. Not to switch the architecture would be another policy. lu --all solely looks at the version numbers, so it lists them.

The dup --from KDE just asks to switch installed packages to the ones provided by the KDE repo. But this does not disable the vendor policy. You are informed, that switching to plasma5-session from KDE will change the packages vendor from openSUSE to obs://build.opensuse.org/KDE and will also require the installed systemsettings5-devel to be removed. You have to decide whether you are fine with this change or not.

The 1st dup either did not touch plasma5-session or the previously installed version was from openSUSE as well. In this casse no policy violation had to be reported.

So up dup dup --from REPO define the job to perform. --allow-vendor-change (or -arch-change or -downgrade) tell which policies should be active or disabled.

So dup --from KDE --allow-vendor-change sould perform the task without complaining.

The global default for the vendor policy is defined in /etc/zypp/zypp.conf (for dup and not-dup commands). They are overruled by the --[no-]allow-vendor-change switch.

[mpeter50]

@mpeter50 If zypper dup is used for an update, there should be a solver-testcase at /var/log/updateTestcase-YYYY-MM-DD-hh-mm-ss (date and time of the update). Please pack the testcase directory together with the /var/log/zypper.log and save it somewhere in case we need it.

  cd /tmp
  tar cvjf zypper-testcase.bz2 /var/log/updateTestcase-YYYY-MM-DD-hh-mm-ss /var/log/zypper.log

I'll do that as soon as I will boot into that installation the next time, but I'm afraid it's too late. Before writing this issue, I've read #290, and wanted to check a few earlier update runs. I wasn't able to read the first dup logs from Sunday, because the zypper log [PID] command exited with an error code.

Though it was a somewhat fresh installation, I can possibly reproduce it. Haven't tried it though, because I don't have enough storage space for a new virtual machine, and also, I've installed with ext4, so there are no snapshots.

---

On the second comment, thanks for the explanation. A few follow-up questions:

This is why zypper lu tells that there are packages with a higher version number, but they won't be installed per default because they violate e.g. the vendor policy. Not to switch the architecture would be another policy. lu --all solely looks at the version numbers, so it lists them.

But when I ran zypper list-updates --alow-vendor-change, why doesn't it list the 5 updates? The architecture shouldn't change, and this is not a downgrade, so this can only be hidden because of a vendor change, no? Or is there something else too that can hide the update?

The dup --from KDE just asks to switch installed packages to the ones provided by the KDE repo. But this does not disable the vendor policy.

Oh, so in this example, in the end it wouldn't do that update because it would violate the policy, but it doesn't know that at this point.

So dup --from KDE --allow-vendor-change sould perform the task without complaining.

Why is the --from needed, if priorities are higher in the target repo(s)?

[mlandres]

But when I ran zypper list-updates --alow-vendor-change, why doesn't it list the 5 updates?

We definitely need to work on the zypper output here. Despite it's higher version number a package is not picked, if it violates a policy (arch,vendor,name change) or is involved in a dependency conflict. Looking at the dup conflict it might be due to the need to additionally remove systemsettings5-devel. But in any case zyppers output should give the hint. That's a usability bug we have to fix.

Why is the --from needed, if priorities are higher in the target repo(s)?

The --from is not needed. It just restricts the scope of the dup to a smaller set of packages and leaves the rest of the system unchanged. The --allow-vendor-change is what makes the difference:

 1: installed Foo-1.0-0.x86_64 (suse)

 2: available Foo-1.0-0.x86_64 (suse)
 3: available Foo-1.5-0.x86_64 (Foreign)
 4: available Foo-2.0-0.x86_64 (suse)

2,3,4 are the potential update candidates. If vendor change is allowed, 3 stays in the set. Otherwise 3 is discarded because the installed package is from different vendor. After the set has been pruned, the solver picks the 'best' of the remaining candidates. This is the point where the repo priorities come into play. So in order to be picked due to the higher repo priority, 3 must have survived the pruning.

[mpeter50]

Thanks for the clarifications.

So, to sum it up, the dup left out packages most probably because of package conflicts, and the bug I'm facing is probably a missing conflict resolution prompt, and not counting the conflict into the "skipped" or "error" counter, right?

Besides the mentioned archive file, can I provide anything else so it may be easier to debug the problem?

Also, in the meantime, I think I found out how one can reproduce this problem. It depends on the installed patterns, but I only tested enabling patterns at system installation time, not later in the booted system. This test system was installed by the Leap 15.3 Offline installer. I've also saved the output of the commands that I ran to update KDE, you can find it here. As far as I can tell, this is exactly the state I experienced that I discribed in the issue. Screenshot summary with version differences and kept back packages.

[mlandres]

So, to sum it up, the dup left out packages most probably because of package conflicts, and the bug I'm facing is probably a missing conflict resolution prompt, and not counting the conflict into the "skipped" or "error" counter, right?

No, I don't think so. The initial dup most probably left out the OBS packages because no --allow-vendor-change was set.

Regarding the conflict message: dups job is to find an update candidate and it found one (without vendor-change the openSUSE package, with vendor-change the KDE one) . So there is no reason for any conflict message.

dup --from KDEs job is to find an update candidate within KDE. This is possible only if vendor-change is allowed. So without vendor change a conflict is raised.

The only issue I see is in up (up is not dup!), which should give more details why packages in ...updates will NOT be installed. Up itself will never raise a conflict. It only updates what applies without conflict.

[mpeter50]

No, I don't think so. The initial dup most probably left out the OBS packages because no --allow-vendor-change was set.

Sorry, probably I miscommunicated something. The first time I have ran dup I've used --allow-vendor-change with it, because this page instructed so. My first screenshot is showing the zypper up command's output after running the zypper dup command for the first time.

Regarding the conflict message: dups job is to find an update candidate and it found one (without vendor-change the openSUSE package, with vendor-change the KDE one) . So there is no reason for any conflict message.

In the case of ffmpegthumbs, plasma5-session, plasma5-session-wayland, systemsettings5 and systemsettings5-lang it choose the openSUSE package over the KDE one even with --allow-vendor-change. This lead to the problem that I noticed with systemsettings, because dup replaced most of the KDE packages with the version from the KDE repo, but not these 5.

I see that I was confusing, I'll make a screen recording starting with the installation and showing what I experienced to clear up things. Though it'll probably be relatively low resolution, so it fits into Github's size limit

[mlandres]

@mpeter50 Maybe you can send me the testcase you saved (e.g per mail to ma at suse.com). I'll have a closer look at it.

[mpeter50]

I've sent an email with the testcase. Also, here is a screencast about reproducing the bug: https://user-images.githubusercontent.com/83356418/157926250-59f430d4-1cbb-4f51-bef2-932be5de1fa3.mp4

I've cut off sections where I was just waiting, but I recommend speeding it up in the player.