Installation fails with new key

jrfnl commented 1 month ago

I haven't received any end-user reports of an issue yet, but was testing after a new release myself and am seeing the following error on a new install:

phive install --trust-gpg-keys 689DAD778FF08760E046228BA978220305CD5C32 phpcs

Phive 0.15.2 - Copyright (C) 2015-2024 by Arne Blankerts, Sebastian Heuer and Contributors
Downloading key A978220305CD5C32
Trying to connect to keys.openpgp.org (37.218.245.50)
Successfully downloaded key.

        Fingerprint: 689D AD77 8FF0 8760 E046 228B A978 2203 05CD 5C32

        Juliette Reinders Folmer (Release key for PHPCS) <juliette@phpcodesniffer.com>

        Created: 2024-05-20

[ERROR]    Signature could not be verified
[ERROR]    Unknown packet

Also on an existing install phive update doesn't seem to do anything, while 3.9.2 is installed and 3.10.1 is the latest release.

Background info

The initial release key was created and uploaded to https://keys.openpgp.org/ a while back.
The fingerprint was also added to the install instructions in the README.
When I did the 3.10.0 release last Monday, for some silly reason, the private key could no longer be found, so I created a new key and uploaded the pub version to https://keys.openpgp.org/ and confirmed it.
I've also updated the fingerprint info in the install instructions in the README.

The old instructions were:

phive install --trust-gpg-keys 95DE904AB800754A11D80B605E6DDE998AB73B8E phpcs
phive install --trust-gpg-keys 95DE904AB800754A11D80B605E6DDE998AB73B8E phpcbf

The new instructions are:

phive install --trust-gpg-keys 689DAD778FF08760E046228BA978220305CD5C32 phpcs
phive install --trust-gpg-keys 689DAD778FF08760E046228BA978220305CD5C32 phpcbf

What am I doing wrong ?

theseer commented 1 month ago

I'm not sure what's going on. The "Unknown packet" message seems to come from gnupg, not phive itself.

I also fail to reproduce this on my test system, not having had phpcs installed before:

$ phive install --trust-gpg-keys 689DAD778FF08760E046228BA978220305CD5C32 phpcs

Phive 0.15.2-30-g23171fc - Copyright (C) 2015-2024 by Arne Blankerts, Sebastian Heuer and Contributors
Fetching repository list
Downloading https://phar.io/data/repositories.xml
Downloading https://phars.phpcodesniffer.com/phars/phive.xml
Downloading https://phars.phpcodesniffer.com/phars/phpcs-3.10.1.phar
Downloading https://phars.phpcodesniffer.com/phars/phpcs-3.10.1.phar.asc
Downloading key A978220305CD5C32
Trying to connect to keys.openpgp.org (37.218.245.50)
Downloading https://keys.openpgp.org/pks/lookup?op=get&options=mr&search=0xA978220305CD5C32
Successfully downloaded key.

    Fingerprint: 689D AD77 8FF0 8760 E046 228B A978 2203 05CD 5C32

    Juliette Reinders Folmer (Release key for PHPCS) <juliette@phpcodesniffer.com>

    Created: 2024-05-20

Linking /home/theseer/.phive/phars/phpcs-3.10.1.phar to /tmp/x5/tools/phpcs

Granted, this is not using the last official release of phive but my development state. Can you just try again and see if the problem fixed itself? ;-)

jrfnl commented 1 month ago

@theseer Just tried again, same result 😢

jrfnl commented 1 month ago

Last resort: I've now gone and thrown the complete Windows Users\%username%\.phive directory away.

Once I'd done that, it started working.

Now sure what's up with that, but happy to be your guinea pig if ever you have the time/desire to look into this more deeply.

I can keep a copy of the "problem" version of the .phive directory to test with (or even share a zip of it with you).

If not, I'm fine with closing this ticket as a non-reproducible hickup in the (my) system.

theseer commented 1 month ago

Yes, please send me the folder. I'm curious :)

I'd assume that the gnupg data directory is somehow b0rked - given the above error message is emitted by gnupg and I wouldn't know how we can possibly have done anything wrong here that would work in all other cases ;)

But I still would like to reproduce this myself :) I'm even willing to boot my win11 vm for it.

jrfnl commented 1 month ago

@theseer Done, I've send you an email with a zip of the corrupt directory.

For the record, I'm still using Windows 10, though I doubt that makes any real difference.

theseer commented 1 month ago

Okay, took some time to get a chance to look into this.

Good news: I can reproduce this on my linux installation using your .phive folder:

$ unrar x 2*

UNRAR 7.00 freeware      Copyright (c) 1993-2024 Alexander Roshal

Extracting from 20240523 JRF broken .phive.rar

Creating    .phive                                                    OK
Creating    .phive/gpg                                                OK
Extracting  .phive/gpg/pubring.kbx                                    OK 
Extracting  .phive/gpg/pubring.kbx.lock                               OK 
Extracting  .phive/gpg/pubring.kbx~                                   OK 
Extracting  .phive/gpg/trustdb.gpg                                    OK 
Creating    .phive/http-cache                                         OK
Creating    .phive/http-cache/phar.io                                 OK
Creating    .phive/http-cache/phar.io/_data_repositories.xml-c3d219b64142c3fe4b931dcd97fee1b5bdcbeb0d  OK
Extracting  .phive/http-cache/phar.io/_data_repositories.xml-c3d219b64142c3fe4b931dcd97fee1b5bdcbeb0d/content  OK 
Extracting  .phive/http-cache/phar.io/_data_repositories.xml-c3d219b64142c3fe4b931dcd97fee1b5bdcbeb0d/etag  OK 
Creating    .phive/http-cache/phar.phpunit.de                         OK
Creating    .phive/http-cache/phar.phpunit.de/_phive.xml-fe236d1c4032a9ae4f6e8563c881fd7a313b43b5  OK
Extracting  .phive/http-cache/phar.phpunit.de/_phive.xml-fe236d1c4032a9ae4f6e8563c881fd7a313b43b5/content  OK 
Extracting  .phive/http-cache/phar.phpunit.de/_phive.xml-fe236d1c4032a9ae4f6e8563c881fd7a313b43b5/etag  OK 
Creating    .phive/phars                                              OK
Extracting  .phive/phars/phpcs-3.9.1.phar                             OK 
Extracting  .phive/phars/phpcs-3.9.2.phar                             OK 
Extracting  .phive/registry.xml                                       OK 
Extracting  .phive/repositories.xml                                   OK 
Creating    .phive/gpg/private-keys-v1.d                              OK
Creating    .phive/_tmp_wrk                                           OK
Creating    .phive/_tmp_wrk/private-keys-v1.d                         OK
All OK

Trying to install:

$ phive --home `pwd`/.phive install --trust-gpg-keys 689DAD778FF08760E046228BA978220305CD5C32 phpcs

Phive 0.15.2-30-g23171fc - Copyright (C) 2015-2024 by Arne Blankerts, Sebastian Heuer and Contributors
[ERROR]    An error occurred while processing your request:

          file_get_contents(D:\Users\Juliette\.phive\phars/phpcs-3.9.1.phar): Failed to open stream: No such file or directory

          #0 src/shared/PharRegistry.php(304)
          #1 unknown file(0): PharIo\Phive\Cli\Runner->errorHandler()
          #2 src/shared/PharRegistry.php(304): file_get_contents()
          #3 src/shared/PharRegistry.php(282): PharIo\Phive\PharRegistry->loadPharFile()
          #4 src/shared/PharRegistry.php(77): PharIo\Phive\PharRegistry->nodeToPhar()
          #5 src/shared/repository/LocalRepository.php(28): PharIo\Phive\PharRegistry->getPhars()
          #6 src/services/resolver/LocalAliasResolver.php(31): PharIo\Phive\LocalRepository->getReleasesByRequestedPhar()
          #7 src/services/resolver/RequestedPharResolverService.php(37): PharIo\Phive\LocalAliasResolver->resolve()
          #8 src/commands/install/InstallCommand.php(61): PharIo\Phive\RequestedPharResolverService->resolve()
          #9 src/commands/install/InstallCommand.php(50): PharIo\Phive\InstallCommand->resolveToRelease()
          #10 src/commands/install/InstallCommand.php(45): PharIo\Phive\InstallCommand->installRequestedPhar()
          #11 src/shared/cli/Runner.php(241): PharIo\Phive\InstallCommand->execute()
          #12 src/shared/cli/Runner.php(95): PharIo\Phive\Cli\Runner->execute()
          #13 phive(59): PharIo\Phive\Cli\Runner->run()
          #14 {main}

          Environment: PHP 8.3.8 (on Linux 6.8.11-300.fc40.x86_64)
          Phive Version: 0.15.2-30-g23171fc

          This should not have happened and is most likely a bug.
          Please report it at https://github.com/phar-io/phive/issues, make sure you include
          the full output of this error message. Thank you!

Okay, that makes sense. One cannot simply copy the registry.xml from windows to linux. Especially not when a secondary drive letter is used ;)

We should probably catch that better though..

But the existing phars are probably not relevant for this issue, so let's get rid of the registry and try again:

$ rm .phive/registry.xml 

$ phive --home `pwd`/.phive install --trust-gpg-keys 689DAD778FF08760E046228BA978220305CD5C32 phpcs

Phive 0.15.2-30-g23171fc - Copyright (C) 2015-2024 by Arne Blankerts, Sebastian Heuer and Contributors
Downloading https://phars.phpcodesniffer.com/phars/phive.xml
Downloading https://phars.phpcodesniffer.com/phars/phpcs-3.10.1.phar
Downloading https://phars.phpcodesniffer.com/phars/phpcs-3.10.1.phar.asc
Downloading key A978220305CD5C32
Trying to connect to keys.openpgp.org (37.218.245.50)
Downloading https://keys.openpgp.org/pks/lookup?op=get&options=mr&search=0xA978220305CD5C32
Successfully downloaded key.

    Fingerprint: 689D AD77 8FF0 8760 E046 228B A978 2203 05CD 5C32

    Juliette Reinders Folmer (Release key for PHPCS) <juliette@phpcodesniffer.com>

    Created: 2024-05-20

[ERROR]    Signature could not be verified 
[ERROR]    Unknown packet

Bingo!

As stated before, we do not emit Unknown packet within our code base. So this must be a gnupg issue. Let's see how gnupg deals with this directly:

NUPGHOME=`pwd`/.phive/gpg gpg --quiet --status-fd 1 --lock-multiple --no-permission-warning --no-greeting --exit-on-status-write-error --batch  --no-tty  --with-colons --verify phpcs-3.10.1.phar.asc phpcs-3.10.1.phar
gpg: invalid size of lockfile '/tmp/x7/.phive/gpg/pubring.kbx.lock'
gpg: cannot read lockfile
gpg: can't lock '/tmp/x7/.phive/gpg/pubring.kbx'
[GNUPG:] NEWSIG juliette@phpcodesniffer.com
gpg: Signature made Mi 22 Mai 2024 23:38:06 CEST
gpg:                using RSA key 689DAD778FF08760E046228BA978220305CD5C32
gpg:                issuer "juliette@phpcodesniffer.com"
[GNUPG:] ERRSIG A978220305CD5C32 1 8 00 1716413886 9 689DAD778FF08760E046228BA978220305CD5C32
[GNUPG:] NO_PUBKEY A978220305CD5C32
gpg: Can't check signature: No public key

While it can apparently read the signature file just fine, it has some serious issues with a lockfile - complaining about it being of invalid size, cannot be read and it somehow fails to lock the keyring.

$ du .phive/gpg/pubring.kbx.lock 

0   .phive/gpg/pubring.kbx.lock

0 byte is admittingly rather small ;) - and judging the above error message incorrect. Can you verify that file was 0 bytes on your system as well and this is not just an artifact of "raring" it?

Either way, it's a lockfile. I'm not sure why it would exist when no process is running that could use it. So, let's see what happens when I remove it:

$ rm .phive/gpg/pubring.kbx.lock 

$ GNUPGHOME=`pwd`/.phive/gpg gpg --quiet --status-fd 1 --lock-multiple --no-permission-warning --no-greeting --exit-on-status-write-error --batch  --no-tty  --with-colons --verify phpcs-3.10.1.phar.asc phpcs-3.10.1.phar
[GNUPG:] NEWSIG juliette@phpcodesniffer.com
gpg: Signature made Mi 22 Mai 2024 23:38:06 CEST
gpg:                using RSA key 689DAD778FF08760E046228BA978220305CD5C32
gpg:                issuer "juliette@phpcodesniffer.com"
[GNUPG:] ERRSIG A978220305CD5C32 1 8 00 1716413886 9 689DAD778FF08760E046228BA978220305CD5C32
[GNUPG:] NO_PUBKEY A978220305CD5C32
gpg: Can't check signature: No public key

That looks better. It of course cannot check the signature without importing the pub key first. So that part of the error is expected.

Let's see what phive now thinks:

$ phive --home `pwd`/.phive install --trust-gpg-keys 689DAD778FF08760E046228BA978220305CD5C32 phpcs

Phive 0.15.2-30-g23171fc - Copyright (C) 2015-2024 by Arne Blankerts, Sebastian Heuer and Contributors
Downloading https://phars.phpcodesniffer.com/phars/phive.xml
Downloading https://phars.phpcodesniffer.com/phars/phpcs-3.10.1.phar
Downloading https://phars.phpcodesniffer.com/phars/phpcs-3.10.1.phar.asc
Downloading key A978220305CD5C32
Trying to connect to keys.openpgp.org (37.218.245.50)
Downloading https://keys.openpgp.org/pks/lookup?op=get&options=mr&search=0xA978220305CD5C32
Successfully downloaded key.

    Fingerprint: 689D AD77 8FF0 8760 E046 228B A978 2203 05CD 5C32

    Juliette Reinders Folmer (Release key for PHPCS) <juliette@phpcodesniffer.com>

    Created: 2024-05-20

Linking /tmp/x7/.phive/phars/phpcs-3.10.1.phar to /tmp/x7/tools/phpcs

So, looks like it was the broken lock file.

I do not see how we could reliably detect that within phive as the above error messages are not in the requested colon format the rest of the output is in. I'm not even sure on which file descriptor those come in, but a quick dump within phive doesn't seem to contain this error message.

I'll close this issue as I do not think the engineering required to potentially be able to catch this is worth the effort. But at least we know it's not a bug in phive per se ;)

jrfnl commented 1 month ago

@theseer Thank you so much for looking into this! At least we know what was causing this now. It's a weird one indeed and I agree with your conclusion that Phive does not need to handle this situation (unless more, similar reports start pouring in).

Can you verify that file was 0 bytes on your system as well and this is not just an artifact of "raring" it?

I'd deleted the original problem directory, so I could only check via an un-rar myself and with that I got the 0 byte lock file too. 🤷🏻‍♀️

I'll let you know if ever I run into this or similar issues again, but let's consider this investigation closed.

phar-io / phive

Installation fails with new key #427

Background info