gpg key import - not processing PGP_PASSPHRASE?

[mpollmeier]

I'm having difficulty importing the GPG key on travis.ci. I defined all settings as described, but get this error related to the PGP_SECRET:

gpg: no valid OpenPGP data found.
gpg: read_block: read error: invalid packet
gpg: import from `[stdin]' failed: invalid keyring
gpg: Total number processed: 0

I created that setting on travis.ci as documented, and running echo $PGP_SECRET|base64 --decode|gpg --import locally works just fine. Looking at https://github.com/olafurpg/sbt-ci-release/blob/d042e9a22d7893b5c08bc8ae70cf9102c1f4ff65/plugin/src/main/scala/com/geirsson/CiReleasePlugin.scala#L27 I don't understand how this can work, since it doesn't pass the PGP_PASSPHRASE, which is necessary to import it in the keyring...? That's just a guess, though. Other than that: any idea what I may be doing wrong?

An alternative solution would be to generate a new key every time. E.g. with gpg --batch --generate-key gpg.conf as documented in https://www.gnupg.org/documentation/manuals/gnupg/Unattended-GPG-key-generation.html. That would also simplify the usage of this plugin. If I was to fork and patch this plugin, would you take that contribution?

[olafurpg]

Thanks for reporting! I don't believe the passphrase is needed to import the secret so I suspect the pgp_secret environment variable is incorrect. Have you double checked that it's properly exported and base64 encoded? It normally ends with =.

The Sonatype CTO already friendly pointed out on Twitter that generating a fresh gpg key per project was "against the spirit of signing". Generating a fresh key per CI run would take it even further. I would prefer to keep the current instructions even if it indeed significantly complicates the process.

[mpollmeier]

The Sonatype CTO already friendly pointed out on Twitter that generating a fresh gpg key per project was "against the spirit of signing". Generating a fresh key per CI run would take it even further. I would prefer to keep the current instructions even if it indeed significantly complicates the process.

Fair enough. Btw I never quite understood why they require signing, if any key is ok...

The difference is probably the gpg version. Travis ships v1.4.16, and locally I'm on v2.2.8 (to unlock all pleasures of bleeding edge). The described steps can only work for gpg v1. One difference is that gpg --import requires the key password (only in v2). There must also be a difference in the key format (e.g., gpg v1 has a version header), otherwise travis would take my key. I cannot test at the moment because travis refuses to build my project (even though it builds others fine).

Once I can confirm and get it working I'll send you a PR for the readme.

[mpollmeier]

After more debugging I figured it out. When I set the PGP_SECRET via the travis UI as described, it didn't build the project exactly one last time, and then denied to do anything, even when manually triggered. Sounds super strange, I know. I played around a bit and figured that when I run

travis encrypt PGP_SECRET=`cat gpg.priv|base64`

I get the error data too large - consider using travis encrypt-file or travis env set. So my best guess is that my keys are longer than yours, and travis doesn't like that. You can imagine how painful that process was..

Workaround:

• use travis encrypt-file gpg.priv --add and import the keys in the install section in .travis.yml using gpg --import gpg.priv
• set PGP_SECRET=foobar so that the plugin doesn't complain

Does that make sense? I can send you a PR that changes the process and readme to cater for this issue and make travis encrypt-file the default. Interested?

[mpollmeier]

Thoughts/opinions on the above? I can make the suggested changes in a fork and use that instead, but I think it would make sense to bring it back into this project...

[olafurpg]

Sorry for the slow responses, just got back from vacation :) I'm sad to hear you hit on so many blockers, we should absolutely improve the installation docs to make this smoother.

Btw I never quite understood why they require signing, if any key is ok...

Same here... ¯_(ツ)_/¯

Maybe the best solution is indeed to generate a key on the fly for every CI run. That would require submitting the public key to gpg.mit.edu, which I prefer to avoid.

I can send you a PR that changes the process and readme to cater for this issue and make travis encrypt-file the default. Interested?

I personally prefer the environment variable workflow over travis encrypt-file since I've found it error-prone to encrypt/decrypt files. Environment variables have worked reliably in my experience and help keep .travis.yml short and simple.

Travis ships v1.4.16, and locally I'm on v2.2.8

Does this mean you need to before_install gpg v2 on Travis? Reading https://www.gnupg.org/faq/gnupg-faq.html#pgp_26 it seems v1 is obsolete but judging by https://gnupg.org/download/ there seem to be no plans to announce an EOL for v1. It's interesting Travis doesn't default to v2.

get the error data too large - consider using travis encrypt-file or travis env set. So my best guess is that my keys are longer than yours, and travis doesn't like that

What is your key size? If this problem can be fixed by declaring a smaller key size (1024 instead of default 2048) then I think that might be a simpler solution than travis encrypt-key.

[olafurpg]

I just got back to my computer (first time since this issue was opened) and it seems I also have gpg v2.2.8 installed on my computer and my keys use the default size 2048. I wonder what the root cause of the problem is 🤔

[mpollmeier]

Interesting. Mine is rsa2048, so that's probably the same. What are the results for these two on your machine?

cat gpg.priv|base64|wc
#      63      63    4851
# my thought was that this is too long for a travis var

travis encrypt PGP_SECRET=`cat gpg.priv|base64`
# data too large - consider using travis encrypt-file or travis env set

[mpollmeier]

Does this mean you need to before_install gpg v2 on Travis? Reading https://www.gnupg.org/faq/gnupg-faq.html#pgp_26 it seems v1 is obsolete but judging by https://gnupg.org/download/ there seem to be no plans to announce an EOL for v1. It's interesting Travis doesn't default to v2.

Upgrading gpg to v2 on travis would have been an option, if it wasn't for the main blocker: Travis doesn't start any new builds as soon as I add the PGP_SECRET variable via the UI. Trying it via travis encrypt as above lead me to thinking that it magically stopped working because the var is too long (and I still believe that).

[olafurpg]

Where does gpg.priv come from? I use gpg --gen-key and then gpg --armor --export-secret-keys $LONG_ID | base64 where $LONG_ID is printed at the end of --gen-key.

[mpollmeier]

Now you didn't follow your own docs - you meant to use --export-secret-keys :P Here's my output. Can you share your's for the commands in https://github.com/olafurpg/sbt-ci-release/issues/13#issuecomment-409895467 ?

gpg --gen
...
pub   rsa2048 2018-08-02 [SC] [expires: 2020-08-01]
      C2574517808201A2823F09CB33FC34A97CD54CB9
uid                      codepropertygraph bot <michael@michaelpollmeier.com>
sub   rsa2048 2018-08-02 [E] [expires: 2020-08-01]
...

gpg --armor --export-secret-keys C2574517808201A2823F09CB33FC34A97CD54CB9 > gpg.priv

[olafurpg]

$ gpg --list-keys
...
pub   rsa2048 2018-08-02 [SC] [expires: 2020-08-01]
      07983C3FAA00765E314AB03BF8272F88BF7E116F
uid           [ultimate] coursier-small bot <olafurpg@gmail.com>
sub   rsa2048 2018-08-02 [E] [expires: 2020-08-01]
$ gpg --armor --export-secret-keys 07983C3FAA00765E314AB03BF8272F88BF7E116F | base64 | pbcopy
<type in passphrase>
gpg: starting migration from earlier GnuPG versions
gpg: porting secret keys from '/Users/ollie/.gnupg/secring.gpg' to gpg-agent
gpg: [don't know]: invalid packet (ctb=2d)
gpg: read_block: read error: Invalid packet
gpg: import from '/Users/ollie/.gnupg/secring.gpg' failed: Invalid keyring

I ignore the error messages in "starting migration from earlier GnuPG versions" to "failed: Invalid keyring". The secret is already copy-pasted in my clipboard.

[olafurpg]

Could it be that > gpg.priv includes the "gpg: starting migration from earlier GnuPG versions" diagnostics?

[mpollmeier]

I don't get these errors, and the gpg.priv looks fine:

-----BEGIN PGP PRIVATE KEY BLOCK-----
...
-----END PGP PRIVATE KEY BLOCK-----

You can probably temporarily move your ~/.gnupg dir to not get them any more.

I'm still keen to hear if you also get that 'data too large' error:

travis encrypt PGP_SECRET=`cat gpg.priv|base64`
# data too large - consider using travis encrypt-file or travis env set

[olafurpg]

I get the same error,

$ travis encrypt PGP_FOO=`cat gpg.priv| base64`
data too large - consider using travis encrypt-file or travis env set

Are you sure you want to run travis encrypt? It seems that command is to "encrypt values for the .travis.yml". We want to store the base64 encoded value as a secret environment variable, the following command succeeds for me

$ travis env set PGP_FOO "$(cat gpg.priv| base64)"
[+] setting environment variable $PGP_FOO

[mpollmeier]

Hmm, you're probably right, I just used encrypt because env set would share the key with the world (which wouldn't really harm in this case but smells a bit fishy). Not sure how to proceed from here - maybe we shouldn't do anything unless others are having the same issue.

To sum up the prose above: as soon as I set PGP_SECRET via travis.ci UI settings, travis didn't trigger any new builds (and didn't tell me why). I was able to reproduce this on multiple repos (using the same key). My workaround is to travis encrypt-file the key and import it in the install section.

[olafurpg]

as soon as I set PGP_SECRET via travis.ci UI settings, travis didn't trigger any new builds (and didn't tell me why)

Weird, I use this setup for all of my projects and I have not experienced that problem. Are you using any Travis beta features?

[mpollmeier]

Are you using any Travis beta features?

Nope. I wouldn't exclude a user error though.

Since I need a few more amendments my current plan is to fork this plugin. Thankfully it's very little code. E.g. I'd like to make it automatically release and tag a new version, e.g. X.Y.Z where X/Y/Z are numbers and Z auto increments. If it works and you'd like it as well, I'm happy to carve out a PR.

[olafurpg]

Sounds good. I'll keep my eyes open if someone reports similar issues as you encountered.

The fork idea sounds interesting, good luck with that!

[mpollmeier]

Here's what I ended up with: https://github.com/ShiftLeftSecurity/sbt-ci-release-early https://blog.shiftleft.io/open-sourcing-release-plugin-sbt-ci-release-early-4dc4180bb777 https://twitter.com/pollmeier/status/1044294452372799489

[olafurpg]

Nice @mpollmeier ! Glad you found a setup you like 👍 The more people release the better 😄

[ChristopherDavenport]

I am having this issue with a project presently. Persisted across 2 keys. I would suspect it being me, but it seems to work correctly on a different repository.

Bringing it up again, as I'm not sure what's going wrong. As the command and the import command this plugin uses works locally.

[ChristopherDavenport]

Turns out the fix was to remove the newlines it comes with.

Pretty easy once figured out though.

For future reference this does it for me.

gpg --armor --export-secret-keys $LONG_ID | base64 | sed -z 's;\n;;g' | xclip -selection clipboard -i

[raboof]

I had the same symptom, but in my case I just accidentally restricted access to the PGP_SECRET variable to the master branch, and if you do that it's not available when building a tag. My bad :smile: