Duplicate gpg keys in key selection menu for message encryption

[CaveSven]

Hi there, I have recently set up neomutt on OSX and, confusingly, in the key selection menu I am presented a choice of keys that are all the same. I sometimes see two seemingly identical keys and sometimes three. It makes no difference which one I choose. I have been trying to understand where this duplication comes from, but so far have failed to figure it out. It is unclear to me which command neomutt uses in order to fetch available keys for an address. Using gpg from the command line, I do not see any duplicate keys.

I would appreciate any hint :-)

Many thanks!

[roccoblues]

NeoMutt uses the gpgme lib internally to handle GPG stuff. There is no simply "command" we run to fetch the keys. Maybe @alejandro-colomar has an idea how to debug this?

[alejandro-colomar]

Can you please paste screenshots of the problem, to make sure I understand it?

I think I know what you mean, but am not sure.

I can try to debug it.

[CaveSven]

I appreciate the offer to help!

[alejandro-colomar]

I assume those red thingies are 3 different emails? Or are they the same?

[CaveSven]

No, they are three times the same address. I cannot figure out why I see multiple entries here. The consequence is that I have to manually confirm each key, even though there really is only one.

[alejandro-colomar]

Does it happen only with some keys, or with all of them? Do you notice any pattern in when/how it happens?

Do you use autocrypt?

What is your gpgme version? What's your neomutt(1) version?

[alejandro-colomar]

And does it happen in other systems with the same keys?

[CaveSven]

I am on osx, Neomutt Version: 20240425 GPGme: 1.23.2 I only notice that the number of duplicates varies with each key, but didn't find a pattern (like e.g. the number of subkeys). I never had this issue on my arch unit installation. I have set crypt_autocrypt=yes in my config.

[alejandro-colomar]

Are you able to run (old) mutt(1) on that system? And does it reproduce the issue?

[alejandro-colomar]

My first guess was that it would have to do with the number of different IDs that a PGP key has.

For example, my key has 3 IDs, and I thought it could appear thrice in that bug.

$ gpg --list-key alx
pub   rsa4096 2022-03-27 [C]
      A9348594CE31283A826FBDD8D57633D441E25BB5
uid           [ultimate] Alejandro Colomar <alx@alejandro-colomar.es>
uid           [ultimate] Alejandro Colomar <alx@kernel.org>
uid           [ultimate] Alejandro Colomar Andres <alx.manpages@gmail.com>
sub   rsa4096 2022-03-27 [S]
sub   rsa4096 2024-02-02 [E] [expires: 2025-02-01]

Could you check if the number of IDs in your cases are consistent with the number of times the keys appear?

[alejandro-colomar]

Let's try this:

git clone git@github.com:gpg/gpgme.git;
cd gpgme;
./autogen.sh;
mkdir .tmp;
cd .tmp;
../configure --enable-maintainer-mode;
make -j4;
make check;
cd tests;
./run-keylist your-email-here

With the different emails that reproduce this, and see if the number of times that the keys appear are the same.

[CaveSven]

Thanks!

1. Keys only have one uid.
2. See attachment.

[alejandro-colomar]

Hmmmm, I think it's more or less this:

I'll check it.

[CaveSven]

I appreciate you looking into this. :-)

[alejandro-colomar]

I've applied this diff:

diff --git a/ncrypt/crypt_gpgme.c b/ncrypt/crypt_gpgme.c
index 020c2392d..064cd349a 100644
--- a/ncrypt/crypt_gpgme.c
+++ b/ncrypt/crypt_gpgme.c
@@ -3122,6 +3122,7 @@ static struct CryptKeyInfo *get_candidates(struct ListHead *hints, SecurityFlags

     while ((err = gpgme_op_keylist_next(ctx, &key)) == GPG_ERR_NO_ERROR)
     {
+fprintf(stderr, "ALX: fpr: #%s#\n", key->fpr);
       KeyFlags flags = KEYFLAG_NO_FLAGS;

       if (key_check_cap(key, KEY_CAP_CAN_ENCRYPT))
@@ -3138,6 +3139,9 @@ static struct CryptKeyInfo *get_candidates(struct ListHead *hints, SecurityFlags

       for (idx = 0, uid = key->uids; uid; idx++, uid = uid->next)
       {
+fprintf(stderr, "ALX: idx: #%d#\n", (int)idx);
+fprintf(stderr, "ALX: uid: #%s#\n", uid->uid);
+getchar();
         k = mutt_mem_calloc(1, sizeof(*k));
         k->kobj = key;
         gpgme_key_ref(k->kobj);
@@ -3269,7 +3273,9 @@ static struct CryptKeyInfo *crypt_getkeybyaddr(struct Address *a,

   if (!oppenc_mode)
     mutt_message(_("Looking for keys matching \"%s\"..."), a ? buf_string(a->mailbox) : "");
+fprintf(stderr, "ALX %s() begin\n", __func__);
   keys = get_candidates(&hints, app, (abilities & KEYFLAG_CANSIGN));
+fprintf(stderr, "ALX %s() end\n", __func__);

   mutt_list_free(&hints);

@@ -3282,6 +3288,7 @@ static struct CryptKeyInfo *crypt_getkeybyaddr(struct Address *a,
   for (k = keys; k; k = k->next)
   {
     mutt_debug(LL_DEBUG5, "  looking at key: %s '%.15s'\n", crypt_keyid(k), k->uid);
+fprintf(stderr, "ALX: %s(): k->uid: #%s#\n", __func__, k->uid);

     if (abilities && !(k->flags & abilities))
     {
@@ -3323,6 +3330,7 @@ static struct CryptKeyInfo *crypt_getkeybyaddr(struct Address *a,

     if (match)
     {
+fprintf(stderr, "ALX: %s(): match\n", __func__, k->uid);
       struct CryptKeyInfo *tmp = crypt_copy_key(k);
       *matches_endp = tmp;
       matches_endp = &tmp->next;
@@ -3333,6 +3341,7 @@ static struct CryptKeyInfo *crypt_getkeybyaddr(struct Address *a,
         a_valid_addrmatch_key = tmp;
     }
   }
+getchar();

   crypt_key_free(&keys);

which allowed me to see the following in stderr:

ALX crypt_getkeybyaddr() begin
ALX: fpr: #7D69474E84028C5CC0C44163BCFD0B018D2658F1#
ALX: idx: #0#
ALX: uid: #наб <nabijaczleweli@nabijaczleweli.xyz>#

ALX: idx: #1#
ALX: uid: #наб <nabijaczleweli@gmail.com>#

ALX: idx: #2#
ALX: uid: #nabijaczleweli <nabijaczleweli@gmail.com>#

ALX: idx: #3#
ALX: uid: #keybase.io/nabijaczleweli <nabijaczleweli@keybase.io>#

ALX crypt_getkeybyaddr() end
ALX: crypt_getkeybyaddr(): k->uid: #наб <nabijaczleweli@nabijaczleweli.xyz>#
ALX: crypt_getkeybyaddr(): k->uid: #наб <nabijaczleweli@gmail.com>#
ALX: crypt_getkeybyaddr(): match
ALX: crypt_getkeybyaddr(): k->uid: #nabijaczleweli <nabijaczleweli@gmail.com>#
ALX: crypt_getkeybyaddr(): match
ALX: crypt_getkeybyaddr(): k->uid: #keybase.io/nabijaczleweli <nabijaczleweli@keybase.io>#

Comparing to

$ ./run-keylist nabijaczleweli@gmail.com
keyid   : BCFD0B018D2658F1
can_cap : esca
has_cap : esca
flags   : de-vs
upd     : 1708099512 (1)
fpr    0: 7D69474E84028C5CC0C44163BCFD0B018D2658F1
caps   0: sca
flags  0: de-vs
fpr    1: D02E3D7AB5F0B9A9096019B8ED46733532B811FB
caps   1: e
flags  1: de-vs
userid 0: наб <nabijaczleweli@nabijaczleweli.xyz>
    mbox: nabijaczleweli@nabijaczleweli.xyz
   email: nabijaczleweli@nabijaczleweli.xyz
    name: наб
   cmmnt: 
 uidhash: 433C571DA700985C8315DB56AB49330BD0BD66A0
     upd: 0 (0)
   valid: unknown
userid 1: наб <nabijaczleweli@gmail.com>
    mbox: nabijaczleweli@gmail.com
   email: nabijaczleweli@gmail.com
    name: наб
   cmmnt: 
 uidhash: 39D95CD2947CC3BB352E81B41FE76EBCC3EED3CE
     upd: 0 (0)
   valid: unknown
userid 2: nabijaczleweli <nabijaczleweli@gmail.com>
    mbox: nabijaczleweli@gmail.com
   email: nabijaczleweli@gmail.com
    name: nabijaczleweli
   cmmnt: 
 uidhash: D574602FFE3C7CE79994589FF5AC4483CEA25A31
     upd: 0 (0)
   valid: unknown
userid 3: keybase.io/nabijaczleweli <nabijaczleweli@keybase.io>
    mbox: nabijaczleweli@keybase.io
   email: nabijaczleweli@keybase.io
    name: keybase.io/nabijaczleweli
   cmmnt: 
 uidhash: 3122847B6CF34FCB5BED687B70B845D83C8A11AA
     upd: 0 (0)
   valid: unknown

and

$ gpg --list-key --list-options show-unusable-uids,show-unusable-subkeys nab
pub   rsa4096 2015-08-23 [SCA] [expires: 2025-08-20]
      7D69474E84028C5CC0C44163BCFD0B018D2658F1
uid           [ unknown] наб <nabijaczleweli@nabijaczleweli.xyz>
uid           [ unknown] наб <nabijaczleweli@gmail.com>
uid           [ revoked] nabijaczleweli <nabijaczleweli@gmail.com>
uid           [ revoked] keybase.io/nabijaczleweli <nabijaczleweli@keybase.io>
sub   rsa4096 2015-08-23 [E] [expires: 2025-08-20]

I see that it all matches: 4 uids. 2 of them revoked, but somehow still available to neomutt(1). And 2 of them don't pass the tests in

https://github.com/neomutt/neomutt/blob/d7060ffc7268a47cc6c5fdeb4fa81806c56ca9dd/ncrypt/crypt_gpgme.c#L3299-L3321

so they are not shown.

In your case, what do you see for gpg --list-key --list-options show-unusable-uids,show-unusable-subkeys YOURKEYHERE?

I expect two uids that match, and maybe more that don't?

I appreciate you looking into this. :-)

Thanks! :-)

BTW, I'm looking for a job; if you know of something interesting, please let me know. http://www.alejandro-colomar.es/cv

[alejandro-colomar]

Also, it would be interesting if you can apply that diff to your neomutt, and run it, to see what it prints to stderr.

[alejandro-colomar]

From what I can see in https://keyserver.ubuntu.com/pks/lookup?search=F0678C4C22A9A7C2&fingerprint=on&op=index, I don't see why it's showing 3 lines in your case, but maybe applying the diff reveals something.

[alejandro-colomar]

Also, it would be interesting if you can apply that diff to your neomutt, and run it, to see what it prints to stderr.

Hi @CaveSven , Did you have the chance to try this (and a few other things)?