search

rpm-software-management / rpm

The RPM package manager
http://rpm.org
Other
484 stars 358 forks source link

urpmi --no-verify is broken by rpm checking on its own with rpm-sequoia #3142

Open soig opened 1 month ago

soig commented 1 month ago

We are hitting issues on Mageia with 4.20.x. The first issue is that signatures being refused

This one seems to be on Mageia :

error: Verifying a signature using certificate 00EDB89585B012A8916F0DF8B742FA8B80420F66 (Mageia Packages [packages@mageia.org](mailto:packages@mageia.org)):

  1. Certificiate B742FA8B80420F66 invalid: certificate is not alive because: The primary key is not live because: Expired on 2012-03-13T12:10:11Z
  2. Key B742FA8B80420F66 invalid: key is not alive because: The primary key is not live because: Expired on 2012-03-13T12:10:11Z

However this break "urpmi --no-verify" which used to behave like rpm --noverify But now only rpm --nosignature can bypass this check, whereas rpm --no-verify used to do the same, which looks like regression ?

Previously, we used to check pkgs in urpmi but it looks like rpm does it on its own now too: http://gitweb.mageia.org/software/rpm/urpmi/tree/urpm/main_loop.pm#n527 Which calls: http://gitweb.mageia.org/software/rpm/urpmi/tree/urpm/main_loop.pm#n129 Which calls: http://gitweb.mageia.org/software/rpm/urpmi/tree/urpm/signature.pm#n40 Which calls: http://gitweb.mageia.org/software/rpm/perl-URPM/tree/URPM.xs#n3306 Which calls rpmReadPackageFile()

See attached logs showing urpmi output with both rpm-4.19 & 4.20 For me, there's 2 issues :

Do we've to set RPMPROB_FILTER_VERIFY when calling rpmtsRun() ? Would it impact urpmi running on older rpm versions (aka no more checks there) ? Or should we kill that check (but then no more informative error in GUI?

Note that we all those upstream patches have been backported: 0001-Ensure-noarch-packages-don-t-get-debuginfo.patch 0001-Fix-noprep-regression-from-introducing-mkbuilddir.patch 0002-Drop-an-accidentally-added-duplicated-test.patch 0003-Make-build-in-place-much-less-of-a-hack-and-also-wor.patch 0001-Fix-incomplete-header-on-plain-src.rpm-build-modes-r.patch 0001-Hammer-in-no-debuginfo-for-noarch-packages-damn-it-r.patch 0001-Fix-regression-on-subpackage-debuginfo-RPMTAG_SOURCE.patch 0001-Fix-a-buildroot-regression-on-an-early-__spec_instal.patch

See you

LOG.urpmi-rpm-4.19.txt LOG.urpmi-rpm-4.20.txt

pmatilai commented 1 month ago

The key management and all hasn't changed in rpm itself, this is rpm-sequoia being far, far stricter about what is accepted than the rpm internal pgp parser ever was. To go with rpm-sequoia, you'll need to keep your keys in shape, like not have keys expired 12 years ago.

The alternative is using the "traditional" PGP parser, it's still available only now split into separate repo (see INSTALL)

soig commented 1 month ago

Well the key is updated on media (=repos in urpmi). So regular people who do upgrades with the installer will get the new key when removing old media, adding new media. But I'm doing online updates for a decade using Cauldron (~ Rawhide for Fedora):

diff -u KEY-* --- KEY-AVAIL2 2024-06-04 17:07:29.192787460 +0200 +++ KEY-INST2 2024-06-04 17:07:37.793998092 +0200 @@ -1,4 +1,4 @@ -pub rsa4096/0xB742FA8B80420F66 2011-02-07 [SCEA] [expires: 2025-12-31] +pub rsa4096/0xB742FA8B80420F66 2011-02-07 [SCEA] [expired: 2012-03-13] 00EDB89585B012A8916F0DF8B742FA8B80420F66 uid Mageia Packages [packages@mageia.org](mailto:packages@mageia.org) -sig 0xB742FA8B80420F66 2019-06-21 [selfsig] +sig 0xB742FA8B80420F66 2011-02-07 [selfsig]

I guess I'm a special case 😅 But I may have to change urpmi.update so that it refresh the pubkey when it changes 😅

soig commented 1 month ago

Looks like even when adding new media, we don't re-import the key: "pubkey 80420f66 already imported" So I'll need to fix that.

Still @pmatilai, since urpmi checks pkgs first, should it set RPMPROB_FILTER_VERIFY when calling rpmtsRun() for real since at that point the user said yes to force ?

pmatilai commented 1 month ago

Yeah rpm doesn't have a meaningful way to update keys because back when I added the "is primary key id present", we didn't know anything about expiry, revokes, subkeys .. nothing. https://github.com/rpm-software-management/rpm/pull/3083 promises to fix that, until then the best you can do is try importing, if it already exists erase the key and try again.

I don't know what urpmi --no-verify does, exactly. The story on rpm side is complicated as there are multiple related and partially overlapping mechanisms: the verify step during transactions is not a "signature check" per se, it's more of a payload validation step. Signatures can be used in there, but aren't by default. If urpmi's --no-verify is supposed to be a direct --nosignature counterpart then see 5ed632ae8a2a4a6de69a0d800340c9dc5d529ec0.

soig commented 1 month ago

So there's two issues, so I'll use 2 parts :

Updating/reimporting a pubkey: Stupid question but : how can one remove a key from rpmdb (through the API, not using rpm -e) ? we've rpmtsImportPubkey() but no remove or refresh API. There's also rpmKeyringAddKey() but no RemoveKey().

Currently the URPM C binding uses rpmtsImportPubkey() https://gitweb.mageia.org/software/rpm/perl-URPM/tree/URPM.xs#n3363 (called from the perl interface : https://gitweb.mageia.org/software/rpm/perl-URPM/tree/URPM/Signature.pm#n82)

➡️ I guess I'll have to run a small transaction removing the offending key before importing. I don't see anything else ? WDYT?

Urpmi --no-verify By default (unless one uses --no-verify), urpmi first checks packages http://gitweb.mageia.org/software/rpm/urpmi/tree/urpm/main_loop.pm#n527 before doing the actual installation: http://gitweb.mageia.org/software/rpm/urpmi/tree/urpm/main_loop.pm#n544

The actual installation basically means we run a series of transactions here http://gitweb.mageia.org/software/rpm/urpmi/tree/urpm/install.pm#n352 my $trans = $db->create_transaction; $trans->check $trans->order $trans->run Which are defined in URPM binding : https://gitweb.mageia.org/software/rpm/perl-URPM/tree/URPM.xs#n2770 : respectively rpmtsCheck(), rpmtsOrder() and rpmtsRun()

But before installing, as said above, unless using --no-verify, we'll do some tests : http://gitweb.mageia.org/software/rpm/urpmi/tree/urpm/main_loop.pm#n129 which really happens here: http://gitweb.mageia.org/software/rpm/urpmi/tree/urpm/signature.pm#n40 For each package we call verify_signature() which is there in the URPM binding: https://gitweb.mageia.org/software/rpm/perl-URPM/tree/URPM.xs#n3306 where we use rpmReadPackageFile() to check each package file.

So at this point we already checked the pkgs for signatures. If in interactive mode (either CLI or GUI) we ask the user if he wants to abort and force the installation. Previously, this worked fine, but now rpmlib checks again the pkgs after we already did it and it doesn't know that the user actually said "frak this, install anyway".

So I think that in urpmi context, we want to prevent rpmlib to check again the pkgs since we already did it. ➡️ _Does passing RPMPROB_FILTER_VERIFY would be enough to achieve this_ when calling rpmtsRun()?

pmatilai commented 1 month ago

I guess I'll have to run a small transaction removing the offending key before importing.

Yes, there's nothing better at the moment.

Does passing RPMPROB_FILTER_VERIFY would be enough to achieve this when calling rpmtsRun()?

Maybe, but again that verify step is NOT about signature verification as such, it could be package whose (header) signature is perfectly valid but truncated payload, or such. For "frak the signatures", IIRC you want to set rpmtsSetVfyFlags() to same as rpmtsSetVSFlags().

Note that letting verify do its job has more subtle side-effects too: installations only show as verified in the auditing log if rpm itself verified it.

soig commented 1 month ago

I guess I'll have to run a small transaction removing the offending key before importing.

Yes, there's nothing better at the moment.

I've a working patch for that. Though that's not ideal b/c when adding a whole set of media/repos (core, nonfree, tainted -- each one existing as regular, updates, updates_testing), that mean adding/removing the same key for all. Is there a way to get the expiry date with rpm API?

BTW other packagers are throwing stones at mea, saying there's a difference between an expired key and a revoked key : a signature that was valid yesterday doesn't suddenly become invalid today, just because the key expired!"

Does passing RPMPROB_FILTER_VERIFY would be enough to achieve this when calling rpmtsRun()?

Maybe, but again that verify step is NOT about signature verification as such, it could be package whose (header) signature is perfectly valid but truncated payload, or such. For "frak the signatures", IIRC you want to set rpmtsSetVfyFlags() to same as rpmtsSetVSFlags().

Note that letting verify do its job has more subtle side-effects too: installations only show as verified in the auditing log if rpm itself verified it.

That would only be set up if the urpmi verify pass failed and the user said "go on anyway" or if he sets up the no-verify option in /etc/urpmi/urpmi.cfg ot used the --no-verify option, so in regular cases that won't be an issue

pmatilai commented 1 month ago

BTW other packagers are throwing stones at mea, saying there's a difference between an expired key and a revoked key : a signature that was valid yesterday doesn't suddenly become invalid today, just because the key expired!"

It gets all a bit filosophical. If it doesn't come invalid at that point, then when does it? And if it doesn't, what's the point of expiry in the first place? It shouldn't be possible to create new signatures with an expired key for one, but I guess expiry should be considered wrt the time of signing instead of "now" I suppose. I kinda thought that's how Sequoia does it, but not sure (tagging @nwalfield for possible comments). To evaluate what actually happens one needs to look at both the key and the signature, which we don't have linked here AFAICS.

soig commented 1 month ago

Yeah but aborting on expired key when we don't have an API to refresh the key yet is an issue. We should first add such an API, then later abort on expired keys. As we say in French, "c'est mettre la charrue avant les boeufs" (=~ "it's putting the cart before the horse")😉 At least checking against signing date rather than now would be better in the current state of the rpms

pmatilai commented 1 month ago

Oh, no disagreement there. This is a case where basically everybody got caught with their pants down: in the first 20+ years of rpm's existence nobody gave much thought to expiry and all that, but then in recent years (even before sequoia) people started running into their ancient SHA1 based keys suddenly stopping to work, first due to FIPS and then more generally distros tried to push for disabling obsoleted crypto but disabling SHA1 got postponed and postponed and postponed for things like google-chrome, and then rpm-sequoia was introduced and pushing those people more strongly to update their signing systems. And when those parties finally updated their keys we realize there's no mechanism for them to update those keys at all. :see_no_evil: