Refresh PGP keys, e.g. when prolonging an expiration time

[xsuchy]

When the package is signed by the GPG key, and the key is prolonged, then GPG does not recognize that the content of the GPG key was changed and refuses to update the package.

This is affecting Copr users and originally reported as https://github.com/fedora-copr/copr/issues/2894

Steps to reproduce:

1. Create repository, sign the package with GPG key expiring in 5 years (this is what we have in Copr, for testing purpose you can lower it to one day :) )
2. wait 5 years
3. Other projects and user usually generate and use new GPG key. In Copr, we instead prolong the key. E.g. see https://filipe.kiss.ink/renew-expired-gpg-key/
4. This DOES NOT change the gpg id.
5. after the old expire date try to install the rpm

You will get:

  1. Certificiate 454724A7D1C452B2 invalid: certificate is not alive
      because: The primary key is not live
      because: Expired on 2022-09-02T17:42:01Z
  2. Key 454724A7D1C452B2 invalid: key is not alive
      because: The primary key is not live
      because: Expired on 2022-09-02T17:42:01Z
error: Verifying a signature using certificate 519B71E71D5251A03A517DF8454724A7D1C452B2 (praiskup_myvpn (None) <praiskup#myvpn@copr.fedorahosted.org>):
  1. Certificiate 454724A7D1C452B2 invalid: certificate is not alive
      because: The primary key is not live
      because: Expired on 2022-09-02T17:42:01Z
  2. Key 454724A7D1C452B2 invalid: key is not alive
      because: The primary key is not live
      because: Expired on 2022-09-02T17:42:01Z
Copr repo for myvpn owned by praiskup                                                   194  B/s | 998  B     00:05    
GPG key at https://download.copr.fedorainfracloud.org/results/praiskup/myvpn/pubkey.gpg (0xD1C452B2) is already installed
The GPG keys listed for the "Copr repo for myvpn owned by praiskup" repository are already installed but they are not correct for this package.
Check that the correct key URLs are configured for this repository.. Failing package is: myvpn-1.3-6.fc38.x86_64
 GPG Keys are configured as: https://download.copr.fedorainfracloud.org/results/praiskup/myvpn/pubkey.gpg
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
Error: GPG check FAILED

The workaround is to:

$ rpm -e gpg-pubkey-d1c452b2-59ac3ee9  # drop the old key

This might be rpm issue. But because DNF has its own db of GPG keys and because this happen primary with DNF I am reporting it here.

[xsuchy]

Note that we will have more and more repositories in Copr that will be hitting this issue, so the priority of this issue will be raised in time.

[praiskup]

A related issue, and related comment: https://github.com/rpm-software-management/rpm-sequoia/issues/50#issuecomment-1682313430 (it did not ring RPM bells those days).

[ppisar]

I guess the problem is that DNF does not periodically redownload a key to refresh it. AFAIK watching expiration time is not enough because there can be different reasons for republish a key. E.g. changing a self-signature hash algorithm.

[praiskup]

I guess the problem is that DNF does not periodically redownload a key to refresh it

Yes, but not just this... even if you re-download the updated key, it has the same key-ID. So if you just tell RPM to install it, it will do nothing - thinking that the key is already installed.

[ppisar]

Copr users recently started to report this issue more frequently. This problem will become more frequent as Copr repositories reach the default expiration time. According to Copr developers resigning RPM packages with a completely new key is not feasible due the volume of the data.

That means DNF needs to come with a mechanism for refreshing the keys. Therefore I'm rising a priority of this issue.

[FrostyX]

Thank you for raising the priority @ppisar. Has there been any progress on this? I've seen multiple reports recently, not only in regard to Copr but for other repositories as well (e.g. OpenRazer).

Reproducer

I wanted to provide a step-by-step, copy-paste reproducer to make your life easier.

Optionally, run a container to not mess up your workstation:

podman run -it quay.io/fedora/fedora:39 bash

Install dependencies

dnf install faketime wget dnf5

Import an expired key

wget https://raw.githubusercontent.com/xsuchy/distribution-gpg-keys/64392d70990254bb10876d68b451c0242dde4f95/keys/copr/copr-agriffis-neovim-nightly.gpg
faketime '-5years' rpm --import copr-agriffis-neovim-nightly.gpg

Enable a repository that uses a prolonged version of this key

wget https://copr.fedorainfracloud.org/coprs/agriffis/neovim-nightly/repo/fedora-39/agriffis-neovim-nightly-fedora-39.repo -P /etc/yum.repos.d/

Try to install a new package from the repository using DNF5

dnf5 install neovim

This will result in a failure:

Transaction failed: Signature verification failed.
PGP check for package "libtree-sitter-0.22.2~git.10.gb7fcf987-1.fc39.x86_64" (/var/cache/libdnf5/copr:copr.fedorainfracloud.org:agriffis:neovim-nightly-53b643565ce4f694/packages/libtree-sitter-0.22.2~git.10.gb7fcf987-1.fc39.x86_64.rpm) from repo "copr:copr.fedorainfracloud.org:agriffis:neovim-nightly" has failed: Problem occurred when opening the package.
Warning: skipped PGP checks for 1 package(s).

This issue wasn't introduced by DNF5, the DNF4 fails as well

dnf4 install neovim

Error for comparison:

error: Verifying a signature using certificate 02AE53EC1C2A0A75EFE90FA529F10B45453D6413 (agriffis_neovim-nightly (None) <agriffis#neovim-nightly@copr.fedorahosted.org>):
  1. Certificiate 29F10B45453D6413 invalid: certificate is not alive
      because: The primary key is not live
      because: Expired on 2024-02-29T15:55:37Z
  2. Key 29F10B45453D6413 invalid: key is not alive
      because: The primary key is not live
      because: Expired on 2024-02-29T15:55:37Z

Expected behavior

I guess the problem is that DNF does not periodically redownload a key to refresh it.

I was thinking about a different solution. When DNF is installing a package from a completely new repository, it asks if I trust its key. Personally, I would expect to get the same dialog when installing a package that was signed by a prolonged known key, e.g.

Importing PGP key 0x453D6413:
 Userid     : "agriffis_neovim-nightly (None) <agriffis#neovim-nightly@copr.fedorahosted.org>"
 Fingerprint: 02AE53EC1C2A0A75EFE90FA529F10B45453D6413
 From       : https://download.copr.fedorainfracloud.org/results/agriffis/neovim-nightly/pubkey.gpg
Is this ok [y/N]:

[FrostyX]

One thing worth mentioning - this is an uncharted territory, even RPM doesn't behave properly.

https://github.com/rpm-software-management/rpm/issues/2577 TL;DR After rpm --import old.gpg running rpm --import new.gpg does nothing.

I am not sure whether DNF implementation is blocked by this bug, or if DNF has its own logic for this.

[j-mracek]

Thank you very much for the reproducer.

It looks like that the problem is already in RPM (problem is reproducible also with DNF4).

I had imported keys faketime '-5years' rpm --import copr-agriffis-neovim-nightly.gpg I've downloaded renewed key wget https://download.copr.fedorainfracloud.org/results/agriffis/neovim-nightly/pubkey.gpg. Then rpm --import pubkey.gpg. RPM does not report anything but the key haven't been refreshed.

[pmatilai]

Yes, rpm --import has no concept of refreshing existing keys, never had. The only way to refresh keys is to erase the old one first. See https://github.com/rpm-software-management/rpm/issues/2577

[j-mracek]

@pmatilai Thank you for the link

[j-mracek]

Even when RPM will somehow manage to import renewed key we need to modify DNF4 because when a key is already in RPMDB we do not try to import it.

base.py:

            for info in keys:
                # Check if key is already installed
                if misc.keyInstalled(self._ts, info.rpm_id, info.timestamp) >= 0:
                    msg = _('GPG key at %s (0x%s) is already installed')
                    logger.info(msg, keyurl, info.short_id)
                    continue

[pmatilai]

We just received a surprise development on this on the rpm side: https://github.com/rpm-software-management/rpm/pull/3083

So you might want to hold off dnf5 work on this until that lands.

[jan-kolarik]

After discussing with the RPM team the future of the above mentioned RPM PR, we concluded that it is not sufficient to solve the issue, as follow-up work in rpm-sequoia is still needed, with an unknown timeline for landing such a patch in Fedora releases.

Therefore, for now, we plan to continue with the workaround approach based on this existing PR for both dnf4 and dnf5.

[pmatilai]

Just FYI, the relevant work has now landed in rpm-sequoia (and rpm): https://github.com/rpm-software-management/rpm-sequoia/pull/70 https://github.com/rpm-software-management/rpm/pull/3202

But, that still only works in rpm git master and I don't know if it'll be backported because the upstream implementation relies on C++ stuff already...

[FrostyX]

With all the building blocks already merged in rpm and rpm-sequoia, do you think we can get this implemented and enabled by default in DNF5 for F42?