cameronkerrnz
commented
3 years ago Ah, that was easy to find, the fix would appear to be to add --rfc4880 to https://github.com/mozilla/sops/blob/master/pgp/keysource.go#L65-L74
Open cameronkerrnz opened 3 years ago
cameronkerrnz
commented
3 years ago Ah, that was easy to find, the fix would appear to be to add --rfc4880 to https://github.com/mozilla/sops/blob/master/pgp/keysource.go#L65-L74
cameronkerrnz
commented
3 years ago pull request #910 submitted
jarrettprosser
commented
3 years ago I also came across this issue when using sops 3.7.1/gpg 2.3.1 on Mac to encrypt files, trying to decrypt files in Ubuntu with gpg 2.2.20. Glad to see there's a fix in the pipeline!
EdgeJ
commented
3 years ago Until #924 can be merged up, there's another workaround I've found to this issue. The main problem with compatibility between RFC4880 and RFC4880bis is in how the keys themselves are generated. I've found that when generating new keys to use with SOPS on a Mac with GPG v2.3, passing the --rfc4880 flag when generating the key itself eliminates this problem (effectively emulating the way keys are generated with GPG 2.2). You can also edit already existing keys to change their preferences to match the RFC4880 standard, if you desire to keep the same fingerprint. This, of course, requires re-encrypting all secrets with the new key.
cameronkerrnz
commented
3 years ago [...] use with SOPS on a Mac with GPG v2.3, passing the
--rfc4880flag when generating the key [...]
Good to know that this affects GPG v2.3 on a Mac as well, and is not specific to Debian Buster. Thanks.
badrow
commented
3 years ago Same problem with Fedora 35 which uses gpg version 2.3.3
neoakris
commented
2 years ago This may be controversial, but it might be worth giving serious consideration of the next version of sops just dropping support for gpg entirely.
gpg and age seem to solve the exact same use case, but age seems easier to maintain, test, install, and is more likely to be bug free / stay bug free since it's written in go vs written in c.
update: just realized my idea is stupid, AGE's crypto's are safe + better UX, but not recognized by NIST so it's not automatically best in 100% of cases.
neoakris
commented
2 years ago Tip for non-interactive copy pasteable command to generate gpg key pair that works consistently on both federa 35++ / centos 9 (gpg 2.3.x) and centos 8 (gpg 2.2.x):
gpg --list-secret-keys
gpg --batch --full-generate-key --rfc4880 --allow-weak-key-signatures <<EOF
%no-protection
Key-Type: RSA
Key-Length: 4096
Subkey-Type: RSA
Subkey-Length: 4096
Expire-Date: 0
Name-Real: my-gpg-2.0.x_and_2.2.x-compatible-test-key-pair
Name-Comment: Test
EOF
gpg --list-secret-keysNote: When using gpg 2.3.x in non-interactive mode --rfc4880 alone is insufficient. You'll see a failure message and gpg --list-secret-keys will show the key didn't get generated. So gpg 2.3.x needs both flags flags --rfc4880 --allow-weak-key-signatures added
I also verified that the exact same non-interactive generation command can be copy pasted into Centos 8, so you can have a consistent experience.
update: the above is a poor workaround
gpg --list-secret-keys
(showed me the fingerprint)
export fp=E75A046BFE318854C4BF7CB54B84FC7893E24710
gpg --armor --export $fp | gpg --list-packets --verbose
(mentions algo 1(RSA & RSA) and digest algo 2(SHA-1)), I'll try to find a better copy paste-able workaround
neoakris
commented
2 years ago Improved version tested against CentOS 9 (gpg --version 2.3.4) & CentOS 8 (gpg --version = 2.2.20) vagrant boxes
# Notice:
# By default, gpg 2.3.x generated keys, will generate errors when
# consumed by clients running gpg 2.0.x - 2.2.x
#
# Workaround:
# When the following gpg key gen command can be copy pasted and ran against
# RHEL 8 (which uses gpg 2.2.x) or Mac / RHEL 9 (which uses gpg 2.3.x)
# The resulting generated key will work correctly for gpg 2.0.x - 2.3.x clients
# (Technically the --rfc4880 & --digest-algo flags are only needed for gpg 2.3.x;
# however, they don't hurt when added to gpg 2.2.x, so it's recommended to
# always use them for the sake of consistency)
# gpg 2.0.x users (RHEL 7 & Amazon Linux) substitute --full-generate-key for --gen-key
#
gpg --batch --full-generate-key --rfc4880 --digest-algo sha512 --cert-digest-algo sha512 <<EOF
%no-protection
# %no-protection: means the private key won't be password protected
# (no password is a fluxcd requirement, it might also be true for argo & sops)
Key-Type: RSA
Key-Length: 4096
Subkey-Type: RSA
Subkey-Length: 4096
Expire-Date: 0
Name-Real: Example
Name-Comment: Example
EOF
ajdergute
commented
2 years ago `#%no-protection: means the private key won't be password protected
This doesn't hold true for sops and I recommend to remove this from your template.
ganto
commented
8 months ago This story goes into a new round:
If one has a GnuPG 2.4 version then even the --rfc4880 flag doesn't disable AEAD support in a new GPG key. I'm not fully sure if intended or not. As a positive side effect the removal of rfc4880bis support as done in e.g. Fedora keeps this flag from "working correctly" also with a recent GPG version.
Once AEAD is supported by the key it's prefered over the old algorithm:
AEAD is a modern and faster way to do authenticated encryption than the old MDC method. [...] The MDC is always used unless the keys indicate that an AEAD algorithm can be used in which case AEAD is used. But note: If the creation of a legacy non-MDC message is exceptionally required, the option --rfc2440 allows for this.
This eventually results in old GPG releases (especially 2.2.x found in e.g. RHEL 8) from not being able to decrypt a file anymore as discussed in this thread.
If with GPG 2.4 a key should be created that is compatible with the old GPG releases one must explicitly remove the AEAD feature as described in the Arch Linux wiki page linked above or discussed in https://github.com/keybase/keybase-issues/issues/4025
tyrannosaurus-becks
commented
4 months ago Thanks for this ticket!
Just to state the workaround I used in shorter terms - I found that:
The problem disappeared. I tested that the file was usable across both Mac and Linux, with sops 3.7, 3.8, and 3.9, and it was.
I'm in the unhappy situation of trying to get my GPG keypair from Windows bind-mounted into a Docker container so I can use ksops (Kustomize-SOPS) on Windows (because kustomize+ksops on Windows is not a happy path, although sops works wonderfully well)
If I take a SOPS encrypted file (encrypted with just GPG; haven't tried anything else), and attempt to decrypt it in a container that has GPG but does not have my keypair, then I get a very unhelpful error message.
The same command works well from outside the container on Windows where my key material is present (but note that this key is not present on any keyserver, as it was generated just for a demonstration purpose).
With debugging:
Testing with just some gpg, I see the problem seems to be there, and makes me think that sops suffers from gpg compatibility issues, particularly when a GPG key was generated on a newer (2.3 in my case) version of GPG, and also used on earlier versions (2.2 in my case)
From Windows (where I generated this keypair), encrypting a simple message to myself:
Decrypting the same, on the same machine (to test the round-trip)
I can verify that text.txt.out matches test.txt.in
This very same file text.txt.enc, being in a directory that is bind-mounted into a Debian Buster container with an earlier version of GPG:
A packet tag of 20 seems new (https://datatracker.ietf.org/doc/html/rfc4880#section-4.3 stops at 19), and tag 20 seems to be https://tools.ietf.org/id/draft-ietf-openpgp-rfc4880bis-06.html#rfc.section.5.16, and this would tally with the Note in the output:
If I encrypt a document in GPG 2.2, then I can happily decrypt it in GPG 2.2 and in GPG 2.3. So its not a matter of where the key is created, but the (implicit) settings used at encrypt-time.
Testing this hypothesis in sops, to evaluate whether encryption by sops (with GPG 2.2) will decrypt with the same version of sops (with GPG 2.3), and I can confirm that it does.
So it seems some consideration needs to be given to how sops should encrypt content with regard to compatibility in decryption, as this could easily lock out team members from decrypting content if it gets encrypted by someone with a newer version.
https://www.gnupg.org/documentation/manuals/gnupg/Compliance-Options.html
... path of least resistance might just be to specify '--rfc4880' if calling out to gpg directly? This makes my problem go away.