Open mcd1992 opened 2 months ago
Without knowledge about the specifics of your setup, I suspect the title does not describe the problem correctly. It should probably be utils.x509.load_cert crashes when confronted with improperly formatted PEM data
. In general, load_cert
should definitely not cause a traceback, while split_pems
should probably work with improperly formatted data.
An initial reaction:
It's generally recommended to dump data (especially strings that can contain newlines) by passing it through the json
filter:
# ...
- append_certs:
- {{ pillar['cacerts']['int_ca'] | json }}
- {{ pillar['cacerts']['root_ca'] | json }}
Otherwise, the rendered YAML can become garbled/invalid (+ you might open yourself up to a template injection attack, depending on the source of the data).
Since your template seems to render fine (when I skip |json
, the YAML is invalid), I assume the pillar values do not contain the properly formatted certificates verbatim (including newlines). The split_pems
function does not fix those though, it expects
-----BEGIN CERTIFICATE-----
MII...
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MII...
...
-----END CERTIFICATE-----
...
and strips anything that's not in between \n?-----BEGIN (CERTIFICATE|...)-----\n[...]\n-----END (CERTIFICATE|...)-----\n?
If the certificates in the pillar are read from a YAML file, they should be defined like:
cacerts:
int_ca: |
-----BEGIN CERTIFICATE-----
MII...
...
-----END CERTIFICATE-----
root_ca: |
-----BEGIN CERTIFICATE-----
MII...
...
-----END CERTIFICATE-----
My pillar data is defined just like that, with the multline |
. And prior to migrating to x509_v2 this worked without | json
or | base64_encode
.
What's weird is when I patch utils/x509.py:load_cert
on the minion and master, to have a debug print for the cert
variable. The master gets a proper PEM string with literal \n
s (not 0x0a) in it but the minion gets spaces or some sort of whitespace instead of \n
.
When I do salt-call pillar.data
on minion and master they both show the proper PEM with newlines. I'll try to replicate with 2 docker containers since I can't replicate with a simple 'masterless' setup.
That's weird. Does | json
fix it (I doubt it)? If not, you can try salt["x509.get_pem_entry"](pillar['cacerts']['int_ca']) | json
.
I can attest that the previous x509
modules ran the certificates that were specified in append_certs
through x509.get_pem_entry
, which fixes improperly formatted PEM entries. Fixing it in the state module is not an option for x509_v2
since it also supports other input types there (base64/hex-encoded data), we would have to adapt the split_pems
function.
| json
does workaround just like | base64_encode
I'll try and get a minimum reproducible setup going for debugging. It may be related somehow to this x509.certificate_managed state being inside a macro as well.
Awesome. Yes, that's a likely culprit.
| json
is not a workaround per se, it is how multiline strings should always be escaped, documented (not in the best location) here: https://github.com/saltstack/salt/blob/cf6c1e10687409519c478ac2594b9f16b9afc5be/salt/states/x509_v2.py#L134-L150
If omitted, the resulting YAML should be unparsable:
[salt.state :4326][CRITICAL][746574] Rendering SLS 'base:x509test' failed: could not find expected ':'; line 17
---
[...]
- subjectAltName: 'DNS: fqdn'
- encoding: pem
- append_certs:
- -----BEGIN CERTIFICATE-----
MIIDODCCAiCgAwIBAgIIbfpgqP0VGPgwDQYJKoZIhvcNAQELBQAwKzELMAkGA1UE
BhMCVVMxDTALBgNVBAMMBFRlc3QxDTALBgNVBAoMBFNhbHQwHhcNMjIxMTE1MTQw <======================
Description x509_v2 certificate_managed append_certs parameter is stripping newlines from PEM certificates.
When using the below state the minion will throw a invalid index error in
salt/utils/x509.py:814
because thepems = split_pems(cert)
is receiving PEM pillar data with its newlines replaced with spaces. On the master split_pems receives proper pillar data with the newlines included. But the minion seems to have newlines replaced with spaces.For now as a workaround I've just changed the append_certs to run the pillar data into base64_encode which works.
Setup
Versions Report
Both master and minion are Arch Linux on 3007.0