Fix for CA certificate not reloading properly

[nsimons]

Overview

This is a bug fix for two scenarios.

142 introduces a cached HTTP client. The TLSConfig of the client is only set at 1) initialization time, and 2) 'config write'. If the CA configuration cannot be fetched at initialization time, then the TLSConfig remains empty for the life time of the Vault process, unless the user does manual actions (e.g. seal/unseal, config write).

Similarly, there is a use case where CA can be read directly from disk. If the CA changes, the TLSConfig is not updated properly.

Design of Change

The HTTP client cert pool is updated before each request towards Kubernetes API Server is made. HTTP connections that exist (are alive, according to KeepAlive) will remain and work. New cert pool is taken into use when a if/new HTTP connections are established.

I found it difficult to add a test case for the scenario in #169. I have a test case for the scenario in #170. Do you think this is enough? I have manually tested the #169 scenario and it works.

A bonus change for free: before we write the config to backend, we try to add the CA certificate to a cert pool. If the certificate is invalid, then an error is produced. This basically fixes https://github.com/hashicorp/vault/issues/17315. Let me know if you want this as a separate PR. The reason why I made this change here is that we also check the return code of certPool.AppendCertsFromPEM() in updateHTTPClient(), so we should in config write as well.

Related Issues/Pull Requests

169

170

Contributor Checklist

[ ] Add relevant docs to upstream Vault repository (=> bug fix without doc change) [ ] Add output for any tests not ran in CI to the PR description (eg, acceptance tests) [x] Backwards compatible

[benashz]

Closing this PR in favor of #173.

Thank you for your contribution to HashiCorp!