Open bifek opened 5 years ago
Here's an even simpler case:
❯ echo -n nonl | od -c
0000000 n o n l
0000004
❯ echo -n nonl | eyaml decrypt --stdin | od -c
0000000 n o n l \n
0000005
For what it's worth, the Puppet lookup()
function appears to be removing the final newlines from encrypted values.
It kinda feels like somebody noticed extra newlines being added at some point in the past, and "fixed" it in the wrong place?
# eyaml encrypt -o string -l xyzzy -f server.key >> /etc/hieradata/common.yaml
#
# od -c server.key | tail -3
0003160 - - E N D R S A P R I V A T
0003200 E K E Y - - - - - \n
0003213
# grep '^xyzzy:' /etc/hieradata/common.yaml | sed '{s/^.*PKCS7,//;s/]$//;}' | base64 -d | openssl smime -decrypt -inkey /etc/puppetlabs/keys/private_key.pkcs7.pem -signer /etc/puppetlabs/keys/public_key.pkcs7.pem -inform der | od -c | tail -3
0003160 - - E N D R S A P R I V A T
0003200 E K E Y - - - - - \n
0003213
# puppet apply --color=false -e '$xyzzy=lookup("xyzzy");notice("<[${xyzzy}]>")' | grep '^-----END' | od -c
0000000 - - - - - E N D R S A P R I
0000020 V A T E K E Y - - - - - ] > \n
0000040
The original file ends with a newline, and the encrypted value ends with a newline, but the string returned by the lookup()
function does NOT end with a newline.
Normal string WITHOUT trailing newline. This looks more or less like what I was expecting, with no newline between testing
and ]>
.
# echo -n testing | eyaml encrypt -o string -l xyzzy2 --stdin >> /etc/hieradata/common.yaml
#
# echo -n testing | od -c
0000000 t e s t i n g
0000007
# grep '^xyzzy2:' /etc/hieradata/common.yaml | sed '{s/^.*PKCS7,//;s/]$//;}' | base64 -d | openssl smime -decrypt -inkey /etc/puppetlabs/keys/private_key.pkcs7.pem -signer /etc/puppetlabs/keys/public_key.pkcs7.pem -inform der | od -c | tail -3
0000000 t e s t i n g
0000007
# puppet apply --color=false -e '$xyzzy=lookup("xyzzy2");notice("<[${xyzzy}]>")' | grep ']>'
Notice: Scope(Class[main]): <[testing]>
Normal string WITH trailing newline. Here I was expecting to see a newline between testing
and ]>
, because the last character of the encrypted value is a newline.
# echo testing | eyaml encrypt -o string -l xyzzy3 --stdin >> /etc/hieradata/common.yaml
#
# echo testing | od -c
0000000 t e s t i n g \n
0000010
# grep '^xyzzy3:' /etc/hieradata/common.yaml | sed '{s/^.*PKCS7,//;s/]$//;}' | base64 -d | openssl smime -decrypt -inkey /etc/puppetlabs/keys/private_key.pkcs7.pem -signer /etc/puppetlabs/keys/public_key.pkcs7.pem -inform der | od -c | tail -3
0000000 t e s t i n g \n
0000010
# puppet apply --color=false -e '$xyzzy=lookup("xyzzy3");notice("<[${xyzzy}]>")' | grep ']>'
Notice: Scope(Class[main]): <[testing]>
Necroing this as at least on my installation this still happens and has caused some hair pulling this morning - but this seems like the last place this is discussed by my googling.
The somewhat terrible but functional way I've been dealing with the inconsistency (and the fact that SSH private keys have to have that newline) is to force eyaml
files to use the YAML block format
my_sensitive_ssh_private_key_data: ENC[PKCS7,....]
becomes
my_sensitive_ssh_private_key_data: >
ENC[PKCS7,....]
I can't say I've dug into the rules for how newlines come across from Puppet lookup - but I'm guessing its stripped the trailing newline by some default rules way-way back in the parser. This block format forces that newline to appear - regardless of the content type (ie: encrypted Sensitive strings or just regular old strings)
Pretty sure that "eyaml decrypt" is wrongly emitting newline.
if we take base64-encoded encrypted value and decode it with openssl, we get correct result:
no newline, as expected.
Even more clear example:
so it decrypts properly w/o \n, it just adds \n when emitting, even if input does not contain ending \n unfortunately, it does that also when output is not terminal, ie. to the file, which breaks binary files.