Wrong cert bundle and key returned in certain conditions

[mollonado]

Wrong cert bundle and key returned in certain conditions This is a continuation of https://github.com/Venafi/vault-pki-backend-venafi/issues/90

I was able to reproduce this bug. To be precise I found a scenario where I can trigger the bug 100% of the time.

I little bit of context and a tiny step back. How I discovered the scenario? Since this was not fixed I've put these two lines in our certificate provisioning system that massages the received cert and key from vault via venafi

KEY_MODULUS=openssl rsa -noout -modulus -in key.pem | openssl md5 CERT_MODULES=openssl x509 -noout -modulus -in cert.pem | openssl md5

If these two don't match this causes the cert renewal process to short-circuit. Consider this a safety fuse in this process. So we use the vault-pki-backend-venafi to fetch new certs for some of our services. We use it with combination of a client side tool that can talk to vault and this we use vault as a kind of proxy when we fetch certs from venafi. What I noticed is that on several of our machines, all part of the same cluster and thus having the same CN for the certificate they request from venafi (but it is different one for every machine) this fuse started firing. This meant that the modulus of the certificate and key was different on every machine part of that cluster. Moreover every time I tried to create a new cert, the cert was the same but the key was different. After some back and forth I found that the certificate was in a messed state in Venafi. What I mean is that the processing of the certificate got into some messy state and it was not in OK status. However, this is the interesting bit:

When we requested a cert via vault-pki-backend-venafi, Vault created the private key itself along with the CSR and sents it to venafi, Venafi returns an old (valid) certificate. It is the same certificate whole processing was in a messy state and it was NOT in an OK state. Vault happily returns both - key and cert bundle, although they do not match. Once we are in this situation there is a 100% reproducibility.

I believe we are hitting a cascade of bugs here.

On Venafi TPP side

• First Venafi should not return an old certificate when we request a new one.
• Second if the certicate state is in not OK, it should return any certificate but throw an error.

On the Vault backend plugin side

• Vault should be able to handle the error. Probably it has some generic error handling, but in this particular case the plugin should be able to understand that this is a server-side error and return it as it is to the client. This will help avoid any confusion.
• The plugin should be able to calculate the modulus of the cert vs the key it has generated as a very rudimentary healthcheck and do this when it receives the bundle. So it does not return an invalid cert and key back to the end user.

How to reproduce

• setup vault with the venafi plugin
• create a certificate via vault. Create another one with the same settings/CN. Manually (this is the tricky part) set the status of the certificate as NOT OK. This will involve some knowledge how to simulated failure on the Venafi side. Since the certificate creation involves several stages, we need to fail one of these stages. The important is that we need to have one of more valid certificates (previous versions) and the current one must be in some kind of failed state.
• try to fetch a new certificate via vault+the plugin. The certificate should be an old version but the key will be new (created by vault).

ENVIRONMENT DETAILS Venafi TTP version - 22.3.0.2139 DB Schema version - 22.3.0.0.0.0 Vault - v1.9.2 vault-pki-backend-venafi - 0.10.1

COMMENTS/WORKAROUNDS The only workaround is to fix the certificate in Venafi. Either click retry or delete it so new certs can be properly created.

[mollonado]

@tr1ck3r @ricrodriguezg fyi.

[luispresuelVenafi]

Hi @mollonado,

what you describe for https://github.com/Venafi/vault-pki-backend-venafi/issues/90 and more, it has already been resolved by latest version of our plugin 0.12.2. You mentioned you are using 0.10.1, could you upgrade to latest version and let us know if your issues still persist? In the meantime I'll remove the "bug" tag until we are sure it is.

[luispresuelVenafi]

The issue described have the symptoms of an already resolved issue in latest version of this plugin. I'm closing this issue as months have passed and we didn't get response from OP