TritonDataCenter / pkgsrc

NetBSD/pkgsrc fork for our binary package repositories
https://pkgsrc.smartos.org/
132 stars 51 forks source link

Unable to verify GPG-signed packages #288

Closed jdwhite closed 1 year ago

jdwhite commented 3 years ago

Fresh macOS Catalina install, following bootstrap instructions from https://pkgsrc.joyent.com/install-on-osx/.

Short version: after bootstrapping, installing gpg2 from the Joyent binary package repo, and importing the public signing key into a keychain pointed to by GPG_KEYRING_VERIFY in pkg_install.conf I am unable to validate as the Signature id in the signed package cannot be found:

pkg_add: unable to verify signature: Signature key id 6d94c02e1f32a9ad not found

I've spend the last couple of days trying to validate packages signed with with my own key get the same results (different Signature key id of course). After much fiddling I finally decided that I must be doing something wrong and decided to start fresh with the Joyent bootstrap as a sanity check -- except it also fails for me.

Long version: here are the terminal logs from my install to show precisely what was done.

root@jdwhites-Mac /opt # BOOTSTRAP_TAR="bootstrap-macos14-trunk-x86_64-20200716.tar.gz"
BOOTSTRAP_SHA="395be93bf6b3ca5fbe8f0b248f1f33181b8225fe"
root@jdwhites-Mac /opt # curl -O https://pkgsrc.joyent.com/packages/Darwin/bootstrap/${BOOTSTRAP_TAR}

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 16.8M  100 16.8M    0     0  4253k      0  0:00:04  0:00:04 --:--:-- 4253k
root@jdwhites-Mac /opt # echo "${BOOTSTRAP_SHA}  ${BOOTSTRAP_TAR}" | shasum -c-

bootstrap-macos14-trunk-x86_64-20200716.tar.gz: OK
root@jdwhites-Mac /opt # curl -O https://pkgsrc.joyent.com/packages/Darwin/bootstrap/${BOOTSTRAP_TAR}.asc

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   819  100   819    0     0   2340      0 --:--:-- --:--:-- --:--:--  2340

root@jdwhites-Mac /opt # tar zxpf bootstrap-macos14-trunk-x86_64-20200716.tar.gz -C /
root@jdwhites-Mac /opt # eval $(/usr/libexec/path_helper)

root@jdwhites-Mac /opt # which pkgin
/opt/pkg/bin/pkgin

##
##  BEGIN NOTE
##
At this point, GPG_KEYRING_VERIFY=/opt/pkg/etc/gnupg/pkgsrc.gpg and 
VERIFIED_INSTALLATION=always are set in pkg_install.conf. 
However, /opt/pkg/etc/gnupg/pkgsrc.gpg does not exist (yet).

Even though VERIFIED_INSTALLATION=always I am not prompted to
approve installation of gnupg2 nor any of its dependencies. 
Is this because /opt/pkg/etc/gnupg/pkgsrc.gpg doesn't exist?

##
## END NOTE
##

root@jdwhites-Mac /opt # pkgin in gnupg2
reading local summary...
processing local summary...
processing remote summary (https://pkgsrc.joyent.com/packages/Darwin/trunk/x86_64/All)...
pkg_summary.xz                                                                             100% 2178KB   1.1MB/s   00:02    
calculating dependencies...done.

5 packages to refresh:
  zlib-1.2.11 bzip2-1.0.8 mozilla-rootcerts-1.0.20200529nb1 openssl-1.1.1h editline-3.1.20191231nb2

1 package to upgrade:
  sqlite3-3.33.0nb1

26 packages to install:
  gnupg2-2.2.24 readline-8.0 pinentry-1.1.0nb1 openldap-client-2.4.56 npth-1.6 libusb1-1.0.23nb1 libksba-1.4.0
  libgpg-error-1.39 libgcrypt-1.8.7 libassuan-2.5.3 gnutls-3.6.15 gettext-lib-0.21 p11-kit-0.23.21 nettle-3.6 lzo-2.10
  libunistring-0.9.10 libtasn1-4.16.0 libcfg+-0.7.0 gmp-6.2.1 tcp_wrappers-7.6.4 cyrus-sasl-2.1.27nb1 libiconv-1.14nb3
  ncurses-6.2nb2 user-darwin-20170116 db4-4.8.30nb1 libffi-3.3nb4

5 to refresh, 1 to upgrade, 26 to install
34M to download, 71M to install

proceed ? [Y/n] 
user-darwin-20170116.tgz                                                                   100% 4402     4.3KB/s   00:00    
db4-4.8.30nb1.tgz                                                                          100% 4166KB   4.1MB/s   00:01    
libffi-3.3nb4.tgz                                                                          100%   55KB  55.0KB/s   00:00    
p11-kit-0.23.21.tgz                                                                        100% 3253KB   3.2MB/s   00:01    
nettle-3.6.tgz                                                                             100% 1050KB   1.0MB/s   00:00    
mozilla-rootcerts-1.0.20200529nb1.tgz                                                      100%  568KB 568.4KB/s   00:00    
lzo-2.10.tgz                                                                               100%  141KB 141.3KB/s   00:00    
libunistring-0.9.10.tgz                                                                    100% 1409KB   1.4MB/s   00:01    
libtasn1-4.16.0.tgz                                                                        100%  147KB 147.0KB/s   00:01    
libcfg+-0.7.0.tgz                                                                          100%  162KB 162.2KB/s   00:01    
gmp-6.2.1.tgz                                                                              100%  806KB 806.0KB/s   00:01    
tcp_wrappers-7.6.4.tgz                                                                     100%   85KB  85.4KB/s   00:00    
openssl-1.1.1h.tgz                                                                         100% 5620KB   2.7MB/s   00:02    
cyrus-sasl-2.1.27nb1.tgz                                                                   100%  271KB 271.5KB/s   00:01    
libiconv-1.14nb3.tgz                                                                       100% 1334KB   1.3MB/s   00:01    
ncurses-6.2nb2.tgz                                                                         100% 1540KB   1.5MB/s   00:01    
editline-3.1.20191231nb2.tgz                                                               100%  169KB 169.2KB/s   00:00    
zlib-1.2.11.tgz                                                                            100%  123KB 122.7KB/s   00:00    
sqlite3-3.33.0nb1.tgz                                                                      100% 1399KB   1.4MB/s   00:01    
readline-8.0.tgz                                                                           100%  462KB 462.0KB/s   00:01    
pinentry-1.1.0nb1.tgz                                                                      100%   96KB  96.1KB/s   00:00    
openldap-client-2.4.56.tgz                                                                 100% 1110KB   1.1MB/s   00:01    
npth-1.6.tgz                                                                               100%   18KB  18.4KB/s   00:00    
libusb1-1.0.23nb1.tgz                                                                      100%  102KB 102.3KB/s   00:00    
libksba-1.4.0.tgz                                                                          100%  243KB 242.6KB/s   00:00    
libgpg-error-1.39.tgz                                                                      100%  375KB 375.1KB/s   00:00    
libgcrypt-1.8.7.tgz                                                                        100%  915KB 914.9KB/s   00:01    
libassuan-2.5.3.tgz                                                                        100%  124KB 124.4KB/s   00:00    
gnutls-3.6.15.tgz                                                                          100% 4321KB   4.2MB/s   00:01    
gettext-lib-0.21.tgz                                                                       100%   65KB  64.8KB/s   00:00    
bzip2-1.0.8.tgz                                                                            100%   92KB  92.3KB/s   00:00    
gnupg2-2.2.24.tgz                                                                          100% 4260KB   4.2MB/s   00:01    
installing gnupg2-2.2.24...
libiconv-1.14nb3: copying /opt/pkg/share/examples/libiconv/charset.alias to /opt/pkg/lib/charset.alias
gettext-lib-0.21: copying /opt/pkg/share/examples/gettext/locale.alias to /opt/pkg/share/locale/locale.alias
libgpg-error-1.39: registering info file /opt/pkg/info/gpgrt.info
libassuan-2.5.3: registering info file /opt/pkg/info/assuan.info
libgcrypt-1.8.7: registering info file /opt/pkg/info/gcrypt.info
libksba-1.4.0: registering info file /opt/pkg/info/ksba.info
pinentry-1.1.0nb1: registering info file /opt/pkg/info/pinentry.info
gmp-6.2.1: registering info file /opt/pkg/info/gmp.info
libtasn1-4.16.0: registering info file /opt/pkg/info/libtasn1.info
nettle-3.6: registering info file /opt/pkg/info/nettle.info
libffi-3.3nb4: registering info file /opt/pkg/info/libffi.info
p11-kit-0.23.21: copying /opt/pkg/share/examples/p11-kit/pkcs11.conf.example to /opt/pkg/etc/pkcs11/pkcs11.conf
libunistring-0.9.10: registering info file /opt/pkg/info/libunistring.info
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-client-server-use-case.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-guile.info
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-handshake-sequence.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-handshake-state.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-internals.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-layers.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-logo.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-modauth.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-x509.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls.info
gnutls-3.6.15: registering info file /opt/pkg/info/pkcs11-vision.png
cyrus-sasl-2.1.27nb1: Creating user ``cyrus''
===========================================================================
$NetBSD: MESSAGE,v 1.4 2017/07/26 09:21:10 hauke Exp $

cyrus-sasl-2.1.27nb1 by default does *not* install any authentication plugin
modules.  Until the necessary authentication plugin modules are
installed, e.g. security/cy2-crammd5 or security/cy2-gssapi, you
will probably get errors about "no appropriate mechs available".

The configuration files for applications using the sasl2 libraries
used to be in /opt/pkg/lib/sasl2/, but are now expected in
/opt/pkg/etc/sasl2/. You need to move them after upgrading the pkg.

cyrus-sasl-2.1.27nb1 will use the following directory as the default
path to its plugin modules:

    /opt/pkg/lib/sasl2

The search path for plugin modules can be customized by setting the
SASL_PATH variable to a colon-delimted list of directories in the shell
environment.

To allow plaintext authentication without using the auxprop mechanism, the
package security/cyrus-saslauthd should be installed.

===========================================================================
openldap-client-2.4.56: copying /opt/pkg/share/examples/openldap/ldap.conf to /opt/pkg/etc/openldap/ldap.conf
readline-8.0: registering info file /opt/pkg/info/history.info
readline-8.0: registering info file /opt/pkg/info/readline.info
readline-8.0: registering info file /opt/pkg/info/rluserman.info
gnupg2-2.2.24: registering info file /opt/pkg/info/gnupg.info
refreshing zlib-1.2.11...
upgrading sqlite3-3.33.0nb1...
installing readline-8.0...
readline-8.0: unregistering info file /opt/pkg/info/history.info
readline-8.0: unregistering info file /opt/pkg/info/readline.info
readline-8.0: unregistering info file /opt/pkg/info/rluserman.info
readline-8.0: registering info file /opt/pkg/info/history.info
readline-8.0: registering info file /opt/pkg/info/readline.info
readline-8.0: registering info file /opt/pkg/info/rluserman.info
installing pinentry-1.1.0nb1...
pinentry-1.1.0nb1: unregistering info file /opt/pkg/info/pinentry.info
pinentry-1.1.0nb1: registering info file /opt/pkg/info/pinentry.info
installing openldap-client-2.4.56...
openldap-client-2.4.56: copying /opt/pkg/share/examples/openldap/ldap.conf to /opt/pkg/etc/openldap/ldap.conf
installing npth-1.6...
installing libusb1-1.0.23nb1...
installing libksba-1.4.0...
libksba-1.4.0: unregistering info file /opt/pkg/info/ksba.info
libksba-1.4.0: registering info file /opt/pkg/info/ksba.info
installing libgpg-error-1.39...
libgpg-error-1.39: unregistering info file /opt/pkg/info/gpgrt.info
libgpg-error-1.39: registering info file /opt/pkg/info/gpgrt.info
installing libgcrypt-1.8.7...
libgcrypt-1.8.7: unregistering info file /opt/pkg/info/gcrypt.info
libgcrypt-1.8.7: registering info file /opt/pkg/info/gcrypt.info
installing libassuan-2.5.3...
libassuan-2.5.3: unregistering info file /opt/pkg/info/assuan.info
libassuan-2.5.3: registering info file /opt/pkg/info/assuan.info
installing gnutls-3.6.15...
gnutls-3.6.15: unregistering info file /opt/pkg/info/gnutls-client-server-use-case.png
gnutls-3.6.15: unregistering info file /opt/pkg/info/gnutls-guile.info
gnutls-3.6.15: unregistering info file /opt/pkg/info/gnutls-handshake-sequence.png
gnutls-3.6.15: unregistering info file /opt/pkg/info/gnutls-handshake-state.png
gnutls-3.6.15: unregistering info file /opt/pkg/info/gnutls-internals.png
gnutls-3.6.15: unregistering info file /opt/pkg/info/gnutls-layers.png
gnutls-3.6.15: unregistering info file /opt/pkg/info/gnutls-logo.png
gnutls-3.6.15: unregistering info file /opt/pkg/info/gnutls-modauth.png
gnutls-3.6.15: unregistering info file /opt/pkg/info/gnutls-x509.png
gnutls-3.6.15: unregistering info file /opt/pkg/info/gnutls.info
gnutls-3.6.15: unregistering info file /opt/pkg/info/pkcs11-vision.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-client-server-use-case.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-guile.info
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-handshake-sequence.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-handshake-state.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-internals.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-layers.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-logo.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-modauth.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls-x509.png
gnutls-3.6.15: registering info file /opt/pkg/info/gnutls.info
gnutls-3.6.15: registering info file /opt/pkg/info/pkcs11-vision.png
installing gettext-lib-0.21...
gettext-lib-0.21: copying /opt/pkg/share/examples/gettext/locale.alias to /opt/pkg/share/locale/locale.alias
refreshing bzip2-1.0.8...
installing p11-kit-0.23.21...
p11-kit-0.23.21: copying /opt/pkg/share/examples/p11-kit/pkcs11.conf.example to /opt/pkg/etc/pkcs11/pkcs11.conf
installing nettle-3.6...
nettle-3.6: unregistering info file /opt/pkg/info/nettle.info
nettle-3.6: registering info file /opt/pkg/info/nettle.info
refreshing mozilla-rootcerts-1.0.20200529nb1...
===========================================================================
$NetBSD: MESSAGE,v 1.5 2014/08/10 10:47:42 wiz Exp $

Execute this command to extract and rehash all CA root certificates
distributed by the Mozilla Project, so that they can be used by third
party applications using OpenSSL. It also creates a single file
certificate bundle in PEM format which can be used by applications using
GnuTLS.

    # mozilla-rootcerts install

To mark these certificates as trusted for users of gnupg2, do
the following (assuming default PKG_SYSCONFBASE and a Bourne shell):

    # mkdir -p /usr/pkg/etc/gnupg
    # cd /usr/pkg/etc/gnupg
    # for c in /etc/openssl/certs/*.pem; do
    > openssl x509 -in $c -noout -fingerprint|sed 's|^.*=\(.*\)|\1 S|'
    > done > trustlist.txt
===========================================================================
installing lzo-2.10...
installing libunistring-0.9.10...
libunistring-0.9.10: unregistering info file /opt/pkg/info/libunistring.info
libunistring-0.9.10: registering info file /opt/pkg/info/libunistring.info
installing libtasn1-4.16.0...
libtasn1-4.16.0: unregistering info file /opt/pkg/info/libtasn1.info
libtasn1-4.16.0: registering info file /opt/pkg/info/libtasn1.info
installing libcfg+-0.7.0...
installing gmp-6.2.1...
gmp-6.2.1: unregistering info file /opt/pkg/info/gmp.info
gmp-6.2.1: registering info file /opt/pkg/info/gmp.info
installing tcp_wrappers-7.6.4...
refreshing openssl-1.1.1h...
===========================================================================
The following directories are no longer being used by openssl-1.1.1gnb2,
and they can be removed if no other packages are using them:

    /opt/pkg/etc/openssl/certs

===========================================================================
openssl-1.1.1h: copying /opt/pkg/share/examples/openssl/openssl.cnf to /opt/pkg/etc/openssl/openssl.cnf
installing cyrus-sasl-2.1.27nb1...
===========================================================================
The following users are no longer being used by cyrus-sasl-2.1.27nb1,
and they can be removed if no other software is using them:

    cyrus

===========================================================================
===========================================================================
The following groups are no longer being used by cyrus-sasl-2.1.27nb1,
and they can be removed if no other software is using them:

    mail

===========================================================================
===========================================================================
$NetBSD: MESSAGE,v 1.4 2017/07/26 09:21:10 hauke Exp $

cyrus-sasl-2.1.27nb1 by default does *not* install any authentication plugin
modules.  Until the necessary authentication plugin modules are
installed, e.g. security/cy2-crammd5 or security/cy2-gssapi, you
will probably get errors about "no appropriate mechs available".

The configuration files for applications using the sasl2 libraries
used to be in /opt/pkg/lib/sasl2/, but are now expected in
/opt/pkg/etc/sasl2/. You need to move them after upgrading the pkg.

cyrus-sasl-2.1.27nb1 will use the following directory as the default
path to its plugin modules:

    /opt/pkg/lib/sasl2

The search path for plugin modules can be customized by setting the
SASL_PATH variable to a colon-delimted list of directories in the shell
environment.

To allow plaintext authentication without using the auxprop mechanism, the
package security/cyrus-saslauthd should be installed.

===========================================================================
installing libiconv-1.14nb3...
libiconv-1.14nb3: copying /opt/pkg/share/examples/libiconv/charset.alias to /opt/pkg/lib/charset.alias
installing ncurses-6.2nb2...
refreshing editline-3.1.20191231nb2...
installing user-darwin-20170116...
installing db4-4.8.30nb1...
installing libffi-3.3nb4...
libffi-3.3nb4: unregistering info file /opt/pkg/info/libffi.info
libffi-3.3nb4: registering info file /opt/pkg/info/libffi.info
pkg_install warnings: 0, errors: 0
reading local summary...
processing local summary...
marking gnupg2-2.2.24 as non auto-removable

root@jdwhites-Mac /opt # curl -sS https://pkgsrc.joyent.com/pgp/1F32A9AD.asc | gpg2 --no-default-keyring --keyring /opt/pkg/etc/pkgsrc.gpg --import
gpg: keybox '/opt/pkg/etc/pkgsrc.gpg' created
gpg: directory '/Users/jdwhite/.gnupg' created
gpg: /Users/jdwhite/.gnupg/trustdb.gpg: trustdb created
gpg: key FAE50048FAA66EE0: public key "Joyent Package Signing <pkgsrc@joyent.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1
root@jdwhites-Mac /opt # cd /opt/pkg/etc 
root@jdwhites-Mac etc # ls -l
total 40
drwxr-xr-x  3 root  wheel    96 Jul 16 03:55 gnupg
-rw-r--r--  1 root  wheel  1846 Jul 16 05:01 mk.conf
drwxr-xr-x  3 root  wheel    96 Nov 23 12:28 openldap
drwxr-xr-x  5 root  wheel   160 Nov 23 12:28 openssl
drwxr-xr-x  3 root  wheel    96 Nov 23 12:28 pkcs11
drwxr-xr-x  3 root  wheel    96 Jul 16 05:01 pkg_alternatives
-rw-r--r--  1 root  wheel   207 Jul 16 05:01 pkg_install.conf
drwxr-xr-x  3 root  wheel    96 Jul 16 05:01 pkgin
-rw-r--r--  1 root  wheel  4097 Nov 23 12:35 pkgsrc.gpg
-rw-------  1 root  wheel    32 Nov 23 12:35 pkgsrc.gpg~
drwxr-xr-x  2 root  wheel    64 Nov 23 12:28 sasl2

# root@jdwhites-Mac etc # cat pkg_install.conf 
#GPG_KEYRING_PKGVULN=/opt/pkg/share/gnupg/pkgsrc-security.gpg
GPG_KEYRING_VERIFY=/opt/pkg/etc/pkgsrc.gpg
PKG_PATH=https://pkgsrc.joyent.com/packages/Darwin/trunk/x86_64/All
VERIFIED_INSTALLATION=always

##
## Now /opt/pkg/etc/pkgsrc.gpg exists
##

root@jdwhites-Mac etc # gpg2 --list-keys --no-default-keyring --keyring /opt/pkg/etc/pkgsrc.gpg
/opt/pkg/etc/pkgsrc.gpg
-----------------------
pub   rsa4096 2015-02-03 [SC]
      74C4F303BB457421E42C4DC4FAE50048FAA66EE0
uid           [ unknown] Joyent Package Signing <pkgsrc@joyent.com>
sub   rsa4096 2015-02-03 [E]
sub   rsa4096 2016-01-11 [S]

root@jdwhites-Mac etc # pkgin in nano
calculating dependencies...done.

2 packages to install:
  nano-5.1 ncursesw-6.2

0 to refresh, 0 to upgrade, 2 to install
1438K to download, 3723K to install

proceed ? [Y/n] 
ncursesw-6.2.tgz                                                                           100%  545KB 544.8KB/s   00:01    
nano-5.1.tgz                                                                               100%  894KB 893.7KB/s   00:01    
installing nano-5.1...
BAD PACKET - bit 7 not 1, offset 0!
hi, need to implement 0, offset 2
BAD PACKET - bit 7 not 1, offset 2!
hi, need to implement 0, offset 4
BAD PACKET - bit 7 not 1, offset 36!
hi, need to implement 0, offset 41
recog_userid: not 13
recog_primary_key: not userid
short pubring recognition???
installing ncursesw-6.2...
BAD PACKET - bit 7 not 1, offset 0!
hi, need to implement 0, offset 2
BAD PACKET - bit 7 not 1, offset 2!
hi, need to implement 0, offset 4
BAD PACKET - bit 7 not 1, offset 36!
hi, need to implement 0, offset 41
recog_userid: not 13
recog_primary_key: not userid
short pubring recognition???
pkg_install warnings: 0, errors: 2
pkg_install error log can be found in /var/db/pkgin/pkg_install-err.log

root@jdwhites-Mac etc # cat /var/db/pkgin/pkg_install-err.log                  
---Nov 23 12:28:01: installing gnupg2-2.2.24...
---Nov 23 12:28:13: refreshing zlib-1.2.11...
---Nov 23 12:28:13: upgrading sqlite3-3.33.0nb1...
---Nov 23 12:28:13: installing readline-8.0...
---Nov 23 12:28:14: installing pinentry-1.1.0nb1...
---Nov 23 12:28:15: installing openldap-client-2.4.56...
---Nov 23 12:28:16: installing npth-1.6...
---Nov 23 12:28:16: installing libusb1-1.0.23nb1...
---Nov 23 12:28:16: installing libksba-1.4.0...
---Nov 23 12:28:16: installing libgpg-error-1.39...
---Nov 23 12:28:17: installing libgcrypt-1.8.7...
---Nov 23 12:28:18: installing libassuan-2.5.3...
---Nov 23 12:28:18: installing gnutls-3.6.15...
---Nov 23 12:28:20: installing gettext-lib-0.21...
---Nov 23 12:28:21: refreshing bzip2-1.0.8...
---Nov 23 12:28:21: installing p11-kit-0.23.21...
---Nov 23 12:28:22: installing nettle-3.6...
---Nov 23 12:28:22: refreshing mozilla-rootcerts-1.0.20200529nb1...
---Nov 23 12:28:22: installing lzo-2.10...
---Nov 23 12:28:22: installing libunistring-0.9.10...
---Nov 23 12:28:23: installing libtasn1-4.16.0...
---Nov 23 12:28:24: installing libcfg+-0.7.0...
---Nov 23 12:28:24: installing gmp-6.2.1...
---Nov 23 12:28:25: installing tcp_wrappers-7.6.4...
---Nov 23 12:28:25: refreshing openssl-1.1.1h...
---Nov 23 12:28:27: installing cyrus-sasl-2.1.27nb1...
---Nov 23 12:28:28: installing libiconv-1.14nb3...
---Nov 23 12:28:29: installing ncurses-6.2nb2...
pkg_delete: Couldn't remove /opt/pkg/share/terminfo/32/2621a: No such file or directory
pkg_delete: Couldn't remove /opt/pkg/share/terminfo/68/hp2621a: No such file or directory
pkg_delete: Couldn't remove /opt/pkg/share/terminfo/68/hp70092a: No such file or directory
---Nov 23 12:28:30: refreshing editline-3.1.20191231nb2...
---Nov 23 12:28:30: installing user-darwin-20170116...
---Nov 23 12:28:30: installing db4-4.8.30nb1...
---Nov 23 12:28:31: installing libffi-3.3nb4...
---Nov 23 12:37:35: installing nano-5.1...
pkg_add: unable to verify signature: Signature key id 6d94c02e1f32a9ad not found 
---Nov 23 12:37:35: installing ncursesw-6.2...
pkg_add: unable to verify signature: Signature key id 6d94c02e1f32a9ad not found

root@jdwhites-Mac etc # pkgin in tmux
calculating dependencies...done.

2 packages to install:
  tmux-3.1c libevent-2.1.12

0 to refresh, 0 to upgrade, 2 to install
961K to download, 2369K to install

proceed ? [Y/n] 
libevent-2.1.12.tgz                                                                        100%  645KB 644.7KB/s   00:00    
tmux-3.1c.tgz                                                                              100%  316KB 316.1KB/s   00:00    
installing tmux-3.1c...
BAD PACKET - bit 7 not 1, offset 0!
hi, need to implement 0, offset 2
BAD PACKET - bit 7 not 1, offset 2!
hi, need to implement 0, offset 4
BAD PACKET - bit 7 not 1, offset 36!
hi, need to implement 0, offset 41
recog_userid: not 13
recog_primary_key: not userid
short pubring recognition???
installing libevent-2.1.12...
BAD PACKET - bit 7 not 1, offset 0!
hi, need to implement 0, offset 2
BAD PACKET - bit 7 not 1, offset 2!
hi, need to implement 0, offset 4
BAD PACKET - bit 7 not 1, offset 36!
hi, need to implement 0, offset 41
recog_userid: not 13
recog_primary_key: not userid
short pubring recognition???
pkg_install warnings: 0, errors: 2
pkg_install error log can be found in /var/db/pkgin/pkg_install-err.log
jperkin commented 3 years ago

Just a comment on this part to begin with:

At this point, GPG_KEYRING_VERIFY=/opt/pkg/etc/gnupg/pkgsrc.gpg and VERIFIED_INSTALLATION=always are set in pkg_install.conf. However, /opt/pkg/etc/gnupg/pkgsrc.gpg does not exist (yet).

This isn't correct, the bootstrap should include that file, which contains the public signing key for the packages:

$ tar ztvf bootstrap-macos14-trunk-x86_64-20200716.tar.gz | grep gpg
-rw-r--r--  0 root   wheel    3835 Jul 16 11:55 ./opt/pkg/etc/gnupg/pkgsrc.gpg

Without that file, yes you'd hit problems as there wouldn't be a way to validate the signatures. However I think you might mean /opt/pkg/etc/pkgsrc.gpg that you refer to later?

Even though VERIFIED_INSTALLATION=always I am not prompted to approve installation of gnupg2 nor any of its dependencies. Is this because /opt/pkg/etc/gnupg/pkgsrc.gpg doesn't exist?

Package verification doesn't rely on GnuPG, it uses the netpgpverify library built into the pkg_install tools.

root@jdwhites-Mac etc # cat pkg_install.conf

GPG_KEYRING_PKGVULN=/opt/pkg/share/gnupg/pkgsrc-security.gpg

GPG_KEYRING_VERIFY=/opt/pkg/etc/pkgsrc.gpg`

There should be no need to do this, and there can be incompatibilities between the keys that GnuPG exports and what netpgpverify supports. If you really want to use GPG for verification (which you can, I just don't think there is any benefit in doing so, I spent a while implementing netpgpverify support so that external commands were not necessary) then you'll probably need to also update pkg_install.conf to include GPG=/path/to/gpg2.

I think it'd help to understand what you're trying accomplish. Everything is set up out of the box to automatically validate packages, as you can see from your initial pkgin in gnupg2 it worked fine (the packages wouldn't install if the verification failed). If you just want to verify the bootstrap tarball then the instructions should provide the commands required for that, and you don't need to change pkg_install.conf. If you want to build your own packages and sign them then that's a little more involved and is described here: https://github.com/joyent/pkgsrc/wiki/pkgdev:signing

Let me know if anything needs further explanation.

jdwhite commented 3 years ago

Just a comment on this part to begin with:

At this point, GPG_KEYRING_VERIFY=/opt/pkg/etc/gnupg/pkgsrc.gpg and VERIFIED_INSTALLATION=always are set in pkg_install.conf. However, /opt/pkg/etc/gnupg/pkgsrc.gpg does not exist (yet).

This isn't correct, the bootstrap should include that file, which contains the public signing key for the packages:

You're correct; my bad. That does explain everything I asked about relating to verification of the installation of gnupg2.

root@jdwhites-Mac etc # cat pkg_install.conf

GPG_KEYRING_PKGVULN=/opt/pkg/share/gnupg/pkgsrc-security.gpg

GPG_KEYRING_VERIFY=/opt/pkg/etc/pkgsrc.gpg`

There should be no need to do this, and there can be incompatibilities between the keys that GnuPG exports and what netpgpverify supports. If you really want to use GPG for verification (which you can, I just don't think there is any benefit in doing so, I spent a while implementing netpgpverify support so that external commands were not necessary) then you'll probably need to also update pkg_install.conf to include GPG=/path/to/gpg2.

Indeed, and even just creating a new keyring and importing your key causes verification issues, so I don't see how gpg2 works here at all. I'd seen various examples about setting GPG to /path/to/gpg2 so I thought that GnuPG was supported for signing/verification. I have no particular preference between gpg2/netpgp.

I think it'd help to understand what you're trying accomplish. Everything is set up out of the box to automatically validate packages, as you can see from your initial pkgin in gnupg2 it worked fine (the packages wouldn't install if the verification failed). If you just want to verify the bootstrap tarball then the instructions should provide the commands required for that, and you don't need to change pkg_install.conf. If you want to build your own packages and sign them then that's a little more involved and is described here: https://github.com/joyent/pkgsrc/wiki/pkgdev:signing

Ultimately, yes, building and signing my own packages is what started me down this road. I've known about your repo for a while but wanted to learn how to do this myself, plus set up my own binary repo (which is another story).

In reading the pkgdev:signing page it mentions using different gnupg2 packages depending on which pkgsrc version you're using. On my package build machine I'm using gnupg2 from pkgsrc-current. Sounds like that may be creating compatibility issues when I export my public signing key -- unsure here.

I was using gpg2 to create my signing key, but is the proper procedure to then export the public key with gpg2 or something else? netpgp says that you can export keys but I see no option to do that. I'm guessing it was either exporting with gpg2 or possibly importing with gpg2 to the pkgsrc.gpg keyring on the client machines that was mangling the public signing key. Would appreciate some clarity here.

Thank you.

jdwhite commented 3 years ago
root@jdwhites-Mac etc # file pkgsrc.gpg gnupg/pkgsrc.gpg 
pkgsrc.gpg:       GPG keybox database version 1, created-at Mon Nov 23 20:35:13 2020, last-maintained Mon Nov 23 20:35:13 2020
gnupg/pkgsrc.gpg: PGP/GPG key public ring (v4) created Tue Feb  3 07:11:36 2015 RSA (Encrypt or Sign) 4096 bits MPI=0xf807edd637e2537d...

Guessing netpgp/netpgpverify doesn't support key box database formats, perhaps.

jdwhite commented 3 years ago

gpg2 manpage talking about pubing.gpg files:

"If this file is not available, gpg defaults to the new keybox format and creates a file pubring.kbx unless that file already exists in which case that file will also be used for OpenPGP keys.

Note that in the case that both files, pubring.gpg and pubring.kbx exists but the latter has no OpenPGP keys, the legacy file pubring.gpg will be used. Take care: GnuPG versions before 2.1 will always use the file pubring.gpg because they do not know about the new keybox format. In the case that you have to use GnuPG 1.4 to decrypt archived data you should keep this file."

Sounds like either use <2.1 GnuPG, find a way to make >=2.1 GnuPG write older keyring format, use netpgp, or gpg 1.x. I think this explains why I saw examples of others using gpg2 in there pkg_install.conf files.

jdwhite commented 3 years ago

I was using gpg2 to create my signing key, but is the proper procedure to then export the public key with gpg2 or something else? netpgp says that you can export keys but I see no option to do that. I'm guessing it was either exporting with gpg2 or possibly importing with gpg2 to the pkgsrc.gpg keyring on the client machines that was mangling the public signing key. Would appreciate some clarity here.

Found netpgpkeys. Unfortunately, both netpgp and netpgpkeys yield a segfault for me for every action. I just built these from pkgsrc-current on Catalina. Same results using packages from the Joyent repo.

jdwhite commented 3 years ago

I finally had success with signing and verifying packages using gpg v1.x in that I could successfully do a pkg_admin gpg-sign-package ... followed by a pkg_info -X .... After that success, if I replace GPG= with gpg2 (changing nothing else) then it doesn't find the secret key. Apparently gpg2 doesn't grok the 8-character version of the key id. Trying the longer version from gpg2 --list-keys doesn't work for signing ether -- can't find the key. I don't care about making it work with gpg2, was just curious since it's my impression is that gpg2 is supposed to work for gpg2 <2.1. Plus, I have a feeling that even if I could sign with gpg2 it would fail to verify due to netpgp stuff.

jperkin commented 3 years ago

Yeh, this is why I specifically have our https://github.com/joyent/pkgsrc-joyent/tree/master/gnupg20 package which retains the 2.0.x branch. I needed it because they broke gpg-agent compatibility in 2.2.x but it also seems like they broke compatibility in other areas too.

You should be able to install the gnupg20 package to get it, and then I think things should work correctly.