search

keybase / keybase-issues

A single repo for managing publicly recognized issues with the keybase client, installer, and website.
902 stars 37 forks source link

Fedora 35 keybase rpm fails upgrade due to incorrect code signing key #4092

Closed lfelia closed 2 years ago

lfelia commented 2 years ago

Message from dnf when upgrading:

GPG key at https://keybase.io/docs/server_security/code_signing_key.asc (0x656D16C7) is already installed The GPG keys listed for the "keybase" repository are already installed but they are not correct for this package. Check that the correct key URLs are configured for this repository.. Failing package is: keybase-5.9.0.20211217212642.29bfd9d39f-1.x86_64 GPG Keys are configured as: https://keybase.io/docs/server_security/code_signing_key.asc

So, waiting on a fix

lfelia commented 2 years ago

The key hasn't changed so perhaps the problem is with the package?

gpg code_signing_key.asc gpg: WARNING: no command supplied. Trying to guess what you mean ... pub rsa4096 2013-11-19 [SC] [expires: 2027-11-11] 222B85B0F90BE2D24CFEB93F47484E50656D16C7 uid Keybase.io Code Signing (v1) code@keybase.io sub rsa4096 2013-11-19 [E] [expires: 2027-11-11]

Espionage724 commented 2 years ago

I've also run into this on Fedora 35 today when trying to update packages. For me, directly installing the latest rpm seems to work fine.

scaronni commented 2 years ago

The key listed at https://keybase.io/docs/server_security/code_signing_key.asc is not the one used to sign the package. Will we get an updated key?

scaronni commented 2 years ago

Actually no, the signature is the same but it does not match?....

# rpm -qpi /var/cache/dnf/fedora-35-x86_64-keybase-0d736cb42e54aa7d/packages/keybase-5.9.0.20211217212642.29bfd9d39f-1.x86_64.rpm | grep -E "Version|Signature"
warning: /var/cache/dnf/fedora-35-x86_64-keybase-0d736cb42e54aa7d/packages/keybase-5.9.0.20211217212642.29bfd9d39f-1.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 656d16c7: NOKEY
Version     : 5.9.0.20211217212642.29bfd9d39f
Signature   : RSA/SHA256, Fri 17 Dec 2021 22:45:19 CET, Key ID 47484e50656d16c7
# rpm -qi keybase | grep -E "Version|Signature"
Version     : 5.8.1.20210930160723.fefa22edc1
Signature   : RSA/SHA256, Thu 30 Sep 2021 18:27:50 CEST, Key ID 47484e50656d16c7
lfelia commented 2 years ago

I've also run into this on Fedora 35 today when trying to update packages. For me, directly installing the latest rpm seems to work fine.

I found this solution also worked for me.

lfelia commented 2 years ago

I've also run into this on Fedora 35 today when trying to update packages. For me, directly installing the latest rpm seems to work fine.

I found this solution also worked for me.

sudo dnf install ./keybase_amd64.rpm

keybase 30 kB/s | 3.3 kB 00:00
Dependencies resolved.

================================================================================

Package Arch Version Repository Size

================================================================================

Upgrading: keybase x86_64 5.9.0.20211217212642.29bfd9d39f-1 @commandline 159 M

Transaction Summary

================================================================================

Upgrade 1 Package

Total size: 159 M Is this ok [y/N]: y Downloading Packages: Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Upgrading : keybase-5.9.0.20211217212642.29bfd9d39f-1.x86_64 1/2 Running scriptlet: keybase-5.9.0.20211217212642.29bfd9d39f-1.x86_64 1/2 Redirecting to /bin/systemctl start atd.service

Cleanup : keybase-5.8.1.20210930160723.fefa22edc1-1.x86_64 2/2 Running scriptlet: keybase-5.8.1.20210930160723.fefa22edc1-1.x86_64 2/2 Verifying : keybase-5.9.0.20211217212642.29bfd9d39f-1.x86_64 1/2 Verifying : keybase-5.8.1.20210930160723.fefa22edc1-1.x86_64 2/2

Upgraded: keybase-5.9.0.20211217212642.29bfd9d39f-1.x86_64

Complete!

jboero commented 2 years ago

I wouldn't install the RPM manually. There is a reason DNF fails on key verification errors. If the package doesn't match signature why is that? I'll wait for an updated package that matches out of repo. Especially for something as sensitive as Keybase.

mumblez commented 2 years ago

I also would prefer not to install the rpm manually!

lfelia commented 2 years ago

I wouldn't install the RPM manually. There is a reason DNF fails on key verification errors. If the package doesn't match signature why is that? I'll wait for an updated package that matches out of repo. Especially for something as sensitive as Keybase.

DNF is still installing the package.

Right now I am more concerned with the 100% processor usage while it runs

lfelia commented 2 years ago

This is why we test on VMs

lfelia commented 2 years ago

I downgraded to keybase-5.8.1.20210930160723.fefa22edc1-1

erAck commented 2 years ago

Fwiw, it doesn't seem to be non-matching signatures, but for whatever reason the key told in the package isn't found by rpm although apparently it is installed. Check presence and details of key with rpm -qi gpg-pubkey-656d16c7 and your currently installed Keybase package with rpm -qi keybase but still a rpm -qip /var/cache/dnf/keybase-d40a41b25b1cd8ba/packages/keybase-5.9.0.20211217212642.29bfd9d39f-1.x86_64.rpm (or wherever the file is cached) gives warning: /var/cache/dnf/keybase-d40a41b25b1cd8ba/packages/keybase-5.9.0.20211217212642.29bfd9d39f-1.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 656d16c7: NOKEY on stderr before the (almost same as installed 5.8.1) details are displayed, note the NOKEY. Specifically the short(!) Key ID is the same, yet not found.

erAck commented 2 years ago

As a side note, comparing key packets of the existing gpg-pubkey-656d16c7 and the one from keybase.io reveals only one difference: the latter has one additional signature on the sub key packet

# off=3939 ctb=89 tag=2 hlen=3 plen=549
:signature packet: algo 1, keyid 47484E50656D16C7
    version 4, created 1384876967, md5len 0, sigclass 0x18
    digest algo 2, begin of digest 19 58
    hashed subpkt 2 len 4 (sig created 2013-11-19)
    hashed subpkt 27 len 1 (key flags: 0C)
    hashed subpkt 9 len 4 (key expires after 4y1d0h0m)
    subpkt 16 len 8 (issuer key ID 47484E50656D16C7)
    data: [4095 bits]

inserted, which is an expired self-signature issued on date of creation, before the common to both signature

# off=4491 ctb=89 tag=2 hlen=3 plen=549
:signature packet: algo 1, keyid 47484E50656D16C7
    version 4, created 1510590784, md5len 0, sigclass 0x18
    digest algo 2, begin of digest bd 3a
    hashed subpkt 27 len 1 (key flags: 0C)
    hashed subpkt 2 len 4 (sig created 2017-11-13)
    hashed subpkt 9 len 4 (key expires after 13y360d0h30m)
    subpkt 16 len 8 (issuer key ID 47484E50656D16C7)
    data: [4095 bits]

That's the only difference.

erAck commented 2 years ago

There seems to be an oddity with the combination of this specific key and F35 rpm gpg-pubkey handling. After exporting the key with rpm -qi gpg-pubkey-656d16c7 >keybase-gpg.asc and deleting the key from the keyring with rpm -e gpg-pubkey-656d16c7 it can not be re-imported with rpmkeys --import keybase-gpg.asc (or rpm --import keybase-gpg.asc) that fails with error: keybase-gpg.asc: key 1 import failed. Same for the current code_signing_key.asc Other rpm gpg-pubkey are not affected.

erAck commented 2 years ago

Puzzle solved.

Importing the key into a temporary $GNUPGHOME and doing gpg --list-sigs gives

pub   rsa4096 2013-11-19 [SC] [expires: 2027-11-11]
      222B85B0F90BE2D24CFEB93F47484E50656D16C7
uid           [ unknown] Keybase.io Code Signing (v1) <code@keybase.io>
sig 3        47484E50656D16C7 2013-11-19  Keybase.io Code Signing (v1) <code@keybase.io>
sig 3        47484E50656D16C7 2017-11-13  Keybase.io Code Signing (v1) <code@keybase.io>
sig        1 6052B2AD31A6631C 2017-11-13  [User ID not found]
sig 3     X  63847B4B83930F0C 2013-11-19  [User ID not found]
sig 3        FBC07D6A97016CB3 2017-11-13  [User ID not found]
sub   rsa4096 2013-11-19 [E] [expires: 2027-11-11]
sig          47484E50656D16C7 2017-11-13  Keybase.io Code Signing (v1) <code@keybase.io>

The signatures of 6052B2AD31A6631C (only trust level 1) and 63847B4B83930F0C (expired) are unusable. Editing the key and deleting them, then exporting as armored and (after having removed the original gpg-pubkey-656d16c7) importing that with rpm gives a usable key.

Update 2022-01-07T16:44+01:00 It's not about trust level or expired signatures, it's they have each a subpacket with the critical bit set, which as not handled is correctly rejected. See below in https://github.com/keybase/keybase-issues/issues/4092#issuecomment-1006415845

So:

export GNUPGHOME=/tmp/tmp-gnupg
mkdir $GNUPGHOME
gpg --import code_signing_key.asc
gpg --edit-key 222B85B0F90BE2D24CFEB93F47484E50656D16C7

Inside gpg> select uid 1 and delete signatures:

1
delsig

prompts for each signature whether to delete it, answer No for all except the two signatures in question, end up with Deleted 2 signatures.

save

to save changes and quit.

gpg --export --armor 222B85B0F90BE2D24CFEB93F47484E50656D16C7 >/tmp/tmp.asc
sudo rpm -e gpg-pubkey-656d16c7
sudo rpm --import /tmp/tmp.asc

should finish without any (error) message. After that dnf upgrade keybase works as expected.

The issue here is that the published Keybase signature key should have only proper signatures itself..

lfelia commented 2 years ago

Still not resolved though. Cannot do a proper update

erAck commented 2 years ago

Make sure you follow the steps exactly. That solved it for me.

Or check that after sudo rpm -e gpg-pubkey-656d16c7 there is no copy of the key left in the keyring, i.e. rpm -q gpg-pubkey-656d16c7 must display package gpg-pubkey-656d16c7 is not installed.

nkbooth commented 2 years ago

I'm aware of how to mash their public key in order to make it match what the RPM was signed with. The bigger issue here is "should we trust a key that signed an RPM that we need to do work on in order for validation to succeed?"

To me, the obvious answer is "absolutely not." If I have to modify/massage/tweak a public key in order to make it match the key that signed a package, then that is a package I don't trust. That's kind of the point.

erAck commented 2 years ago

You did not understand. It's not about tweaking the key to make it match the one that signed the package. The key does match. It is the F35 keyring handling that is now stricter in what it accepts as a currently valid signing key. An expired signature on the key invalidates the key, as does a "I do not fully trust" signature. A signature on the key with the critical bit set on an unhandled subpacket invalidates the signature.

ich199 commented 2 years ago

I think the issue is that those 2 signatures have subpackets marked as critical, which the version of RPM on Fedora 35 does not implement, which causes it to treat the key as invalid, due to this commit

I expect that updating and republishing the key to not have those subpackets marked critical will solve the issue

also reported at https://github.com/keybase/client/issues/24637#issuecomment-998965388

erAck commented 2 years ago

Good find. Who creates such signatures with the critical bit set there wants to make things fail.. according to RFC 4880 5.2.3.1. Signature Subpacket Specification:

Bit 7 of the subpacket type is the "critical" bit. If set, it denotes that the subpacket is one that is critical for the evaluator of the signature to recognize. If a subpacket is encountered that is marked critical but is unknown to the evaluating software, the evaluator SHOULD consider the signature to be in error.

An evaluator may "recognize" a subpacket, but not implement it. The purpose of the critical bit is to allow the signer to tell an evaluator that it would prefer a new, unknown feature to generate an error than be ignored.

heronhaye commented 2 years ago

Hi, we believe we have fixed this issue. Please try upgrading Keybase. Thank you.

peterjanes commented 2 years ago

Doesn't look like it:

# dnf clean all
1345 files removed
# dnf update -y keybase
Fedora 35 - x86_64                              2.8 MB/s |  79 MB     00:27
[...]
keybase                                          56 kB/s |  14 kB     00:00
Dependencies resolved.
================================================================================
 Package     Arch       Version                               Repository   Size
================================================================================
Upgrading:
 keybase     x86_64     5.9.0.20220120174718.95a3939b3a-1     keybase     159 M

Transaction Summary
================================================================================
Upgrade  1 Package

Total download size: 159 M
Downloading Packages:
keybase-5.9.0.20220120174718.95a3939b3a-1.x86_6 3.4 MB/s | 159 MB     00:46    
--------------------------------------------------------------------------------
Total                                           3.4 MB/s | 159 MB     00:46     
keybase                                         8.2 kB/s | 3.0 kB     00:00    
GPG key at https://keybase.io/docs/server_security/code_signing_key.asc (0x656D16C7) is already installed
The GPG keys listed for the "keybase" repository are already installed but they are not correct for this package.
Check that the correct key URLs are configured for this repository.. Failing package is: keybase-5.9.0.20220120174718.95a3939b3a-1.x86_64
 GPG Keys are configured as: https://keybase.io/docs/server_security/code_signing_key.asc
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
Error: GPG check FAILED
heronhaye commented 2 years ago

Hmm... I wonder if the problem is that your key in the system cache is the old version and needs to be updated. Could you try removing the public key from rpm first? https://serverfault.com/questions/899418/how-to-remove-a-yum-repo-gpg-key

heronhaye commented 2 years ago

I think this might also be fixed by reinstalling Keybase from https://keybase.io/app.

ksbex commented 2 years ago

Removing they old GPG key worked for me

heronhaye commented 2 years ago

Sorry about the inconvenience. If there is an automated way to update the key (as mentioned in the comments, we simply removed the expired cross-sig from https://keybase.io/docs/server_security/code_signing_key.asc), let me know and I will do another update.

peterjanes commented 2 years ago

dnf install https://prerelease.keybase.io/keybase_amd64.rpm works.

rpm -e gpg-pubkey-656d16c7 followed by dnf update keybase does as well.

I see that a search for "GPG key * already installed" finds this issue, but you may want to document those steps somewhere more visible, perhaps on the Keybase blog?

heronhaye commented 2 years ago

Thank you @peterjanes. I will update the release notes at https://github.com/keybase/client/releases/tag/v5.9.1.