Bad format of private key

[xelatirdan]

Hello! After downloading key and cert from certwarden I found that nginx can't start with it:

# nginx -t
nginx: [emerg] cannot load certificate key "/etc/certwarden/certs/site.com.key": PEM_read_bio_PrivateKey() failed
nginx: configuration file /etc/nginx/nginx.conf test failed

then I found that issue with blank line in the end of key file before END RSA PRIVATE KEY:

qk39nqADPmTtv4lUPLsr7JFSsJzjLCzoqOwFavTSY0KsUbtNSH6d9C4OVMyoT1LC
jrmY7jFo7qJ4cSr/P6CgDrgliXwKuCooq7UCggEAXeXpt2OeqyqpVaGKz0vwHbNk
rj7k9Czb7A/gvJCohLPZPvQ2fm0wuPOE5U2XeRrOWWxptAYx+8DGYyOORoIyYAD9
KZzkTDgCeKsRBMwUzfixAJNbH3ZpLevRjRbq6BH8IUcO55dOk96XJftesHpADPvh
7yNIHX8eYv9AWByxOnsrEsEC9MUnsQ4hqXYVXrIr76FSv0gRez5og55WhdMCKiNF
QkO3RiV5XEZKttzYfzPR7A4bA581N+ua+PsHOjtKDHRQ2CJhJ7gR9NO6SU7tfoZd
KLgS32uYRxqllvF6lI2XGeqb8T3TpE42HXYu6KierEAFrjpSZMM+yOFv74KRnA==

-----END RSA PRIVATE KEY-----

Certwarden version is 0.21.4

[xelatirdan]

I made 10 certificates and 30% of them with this blank line.

[gregtwallace]

Please manually open your database file with something like https://sqlitebrowser.org/ and in the private_keys table check one of the bad keys and see if the pem field has the additional space blank line.

I have not ever seen this issue and am curious what part of the code the issue might be in.

Was the private key source Generate or Paste PEM ?

[xelatirdan]

Please manually open your database file with something like https://sqlitebrowser.org/ and in the private_keys table check one of the bad keys and see if the pem field has the additional space blank line.

Yes, in SQlite database I see that PEM with blank line.

Was the private key source Generate or Paste PEM ?

We used generate key source.

I suppose that this bug in SSL library or tools whose used for generating PEM. Can you describe how certwarden generate PEM and I will try to google or dig in issue? Thank you.

[gregtwallace]

Are you using the container? If not, what OS / version are you on?

The key generation API calls PostNewKey: https://github.com/gregtwallace/certwarden-backend/blob/master/pkg/domain/private_keys/handlers_post.go

Which calls GeneratePrivateKeyPem: https://github.com/gregtwallace/certwarden-backend/blob/master/pkg/domain/private_keys/key_crypto/generate.go

I'm not entirely sure where a space would be coming from.

[xelatirdan]

Are you using the container? If not, what OS / version are you on?

Yes, we use ghcr.io/gregtwallace/certwarden:latest image on Debian 12.5

[gregtwallace]

I think I might have figured out the issue.

Generate also calls the sanitize function and every 64th rune adds a space. If the key ends at this point an extra line would be added. What I'm not entirely sure about is why it only happens sometimes. I suspect 2s complement encoding of the Big Ints in just the right combination causes this to happen.

https://github.com/gregtwallace/certwarden-backend/blob/master/pkg/domain/private_keys/key_crypto/pem_decode.go#L25

I will modify this ASAP and fix it. Thank you for finding everyone's favorite kind of bug: the kind that only happens randomly sometimes.

If you have one of those "bad" keys that you're not actually using, if you can provide me the PEM block I'd appreciate it so I can use it for testing.

[gregtwallace]

Also, while this will be fixed it also looks like nginx doesn’t do as suggested in the spec.

“Furthermore, parsers SHOULD ignore whitespace and other non-base64 characters and MUST handle different newline conventions.” https://datatracker.ietf.org/doc/html/rfc7468#section-2

That is, the extra white space new line SHOULD have been ignored. Of course, should is not MUST so they’re not technically out of spec either.

[xelatirdan]

If you have one of those "bad" keys that you're not actually using, if you can provide me the PEM block I'd appreciate it so I can use it for testing.

Bad key in attach

blankline.key.pem.zip

[gregtwallace]

I confirmed the issue and it should be fixed. New version is building. If you have keys you want to keep but have the space saved in the db, you can download the key file, delete the key in Cert Warden, and then re-add the key using Paste PEM and the formatting will be corrected.

Thanks again for finding this.