Include client certificate in probe_ssl_earliest_cert_expiry

[patrodyne]

Currently, probe_ssl_earliest_cert_expiry includes the earliest expiration for the server peer certificates. When tls_config and cert_file are used to configure mutual authentication, the cert_file property defines a PEM file containing another certificate, the client certificate. The client certificate, and its peers, have an expiration date too. In order to account for the client certificate expiration(s), either the earliest expiry can be included in probe_ssl_earliest_cert_expiry; or, a new metric named probe_ssl_earliest_client_cert_expiry should be added.

[brian-brazil]

The client cert is not the same as the server cert, so should not go in the same metric.

What exactly is it that you're trying to catch?

[patrodyne]

I need to catch the earliest expiration date of the client certificate chain. The name could be probe_ssl_earliest_client_cert_expiry.

[brian-brazil]

So if your process to keep the cert updated on disk for the blackbox exporter fails?

[patrodyne]

Then my planned alert rule in Prometheus will fire 30 days in advance of expiration of the client certificate:

probe_ssl_earliest_client_cert_expiry{job="WATCH-TCPTLS"} - time() < 86400 * 30

[brian-brazil]

Thinking on this, would we add similar for Prometheus or other binaries which also support client cert auth? We should be consistent, and that gets high cardinality in some of those cases.

[patrodyne]

In my architecture, the blackbox_exporter plus the probe_ssl_earliest_cert_expiry metric solves my certificate alert need nicely. My architecture is cloud based with Pound reverse proxy load balancers to front the back end servers. The Pound servers use Let's Encrypt server certs and our local CA client certificate. One client certificate is shared with a restricted list of external applications and expires periodically.

With blackbox_exporter, I probe the Pound servers over TLS to verify the server cert. The exporter uses the same client cert to connect to the Pound servers. The client cert only needs to be verify by the one exporter. Thus, I recommend limiting the new feature to the blackbox_exporter.

On Tue, Feb 25, 2020, 4:52 AM Brian Brazil notifications@github.com wrote:

Thinking on this, would we add similar for Prometheus or other binaries which also support client cert auth? We should be consistent, and that gets high cardinality in some of those cases.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/prometheus/blackbox_exporter/issues/573?email_source=notifications&email_token=AAJKKQXHX3FF4ZRKUOE5V5TRETS7TA5CNFSM4KYUH5G2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM3JR3Y#issuecomment-590780655, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJKKQRRSBHJSKG35JI7YZLRETS7TANCNFSM4KYUH5GQ .

[brian-brazil]

That sounds like over optimising for this one case a bit, why not have whatever is doing the client cert update push that it worked to a pushgateway?

[patrodyne]

Our client cert update is a manual process. The black box alert will greatly improved our ability to keep it valid.

On Tue, Feb 25, 2020, 09:54 Brian Brazil notifications@github.com wrote:

That sounds like over optimising for this one case a bit, why not have whatever is doing the client cert update push that it worked to a pushgateway?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/prometheus/blackbox_exporter/issues/573?email_source=notifications&email_token=AAJKKQTBJNNWEKLMK6RNLL3REUWLJA5CNFSM4KYUH5G2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM4IKJA#issuecomment-590906660, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJKKQRAWJVQAPZJXQSZSFLREUWLJANCNFSM4KYUH5GQ .

[brian-brazil]

That sounds like something you can cover with a cronjob.

[erikgb]

FYI: I think ribbybibby/ssl_exporter can solve the requested use case.

But it would be nice if Blackbox exporter also had support for this - to avoid installing another runtime component, just to do a very similar job. We still keep Blackbox exporter installed to produce other types of metrics.