[Heartbeat] Check certificates for Browser based Synthetic monitors (step 1 main URL)

paulb-elastic commented 3 years ago

Existing TLS certificate checks for Heartbeat monitors are useful to give an early warning that certificates are going to expire (and give users the time to renew them before they expire which may well impact the availability of the site).

These are not currently available for browser based Synthetics monitors, but the requirement and value that certificate checks provides, is just as valuable for these monitors (arguably more so as more hosts and content are included in browser monitors, with a wider number of hosts needed to deliver it to make up the website).

Use case

As a Synthetic browser based monitoring customer
I want to check the status of TLS certificate for the root URL tested with Synthetics (the same as it does for HTTP checks)
So I get visibility and early warning that the main host has an expiring certificate

This requirement is being split into two phases.

Phase 1 certificate check the host of the main URL on step 1 (this issue)

Feature:

Check the certificate status for the host defined in the URL for the user journey (as shown in the Url column of the Monitors Overview page (if this is accessed via HTTPS)
Show the certificate details in the TLS Certificate column on the Monitors Overview page
The certificate details will also be shown on the Certificates status page, the same as other Heartbeat certificate checks

In this example:

www.elastic.co would be checked and appear in the TLS Certificate column
www.mydomain.com would not be checked (not HTTPS) so nothing would be shown in the TLS Certificate column

Phase 2: a future enhancement to check the certificates of all hosts for a user journey monitor is detailed in this issue.

elasticmachine commented 3 years ago

Pinging @elastic/uptime (Team:Uptime)

botelastic[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

paulb-elastic commented 2 years ago

This is still on the backlog as something we want to add - getting the value of cert checks, into real browser monitoring

botelastic[bot] commented 1 year ago

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

linuschew commented 1 year ago

Would it be possible to re-look into this feature request?

paulb-elastic commented 1 year ago

@linuschew this is something we’d like to add to Synthetics in the future, but is on our backlog whilst we focus on the core capabilities right now

Jaraxal commented 10 months ago

+1 This is still an important feature request for Synthetics.

djkprojects commented 3 months ago

Is the issue going to be fixed? With Elastic moving towards Fleet fixing this would eliminate using heartbeat and Uptime Monitors feature.

paulb-elastic commented 3 months ago

@djkprojects I am not sure I understand your point about Fleet.

This issue is poorly named now, I will update to clarify this is specifically for browser monitors within Synthetics (where this TLS feature is not currently available).

Using Synthetics for HTTP/TCP monitors will include TLS certificate checks today, for example:

Unfortunately we still don't have a date for when this capability will be added to browser monitors.

djkprojects commented 3 months ago

@djkprojects I am not sure I understand your point about Fleet.

This issue is poorly named now, I will update to clarify this is specifically for browser monitors within Synthetics (where this TLS feature is not currently available).

Using Synthetics for HTTP/TCP monitors will include TLS certificate checks today, for example:

Unfortunately we still don't have a date for when this capability will be added to browser monitors.

The certificates don't seem to be working for HTTP/TCP monitors either. We are on 8.11.3. Tested by adding two endpoints, one using TCP Ping and another HTTP Ping both with TLS Configuration enabled. Test results come back successful but the TLS certificates section shows no certificates. The TLS fields are present in the synthetics-tcp/http-* indices. What am I missing here?

We use Private locations which require Elastic Agent hence reference to Fleet, in the past we used heartbeat and Uptime which worked fine.

paulb-elastic commented 3 months ago

Hi @djkprojects thanks for the extra info. I'm setting up an 8.11.3 to try this out too.

Having set up both an TCP and HTTP monitor (to TLS enabled enabled endpoints), do you see nothing in the Synthetics > TLS Certificates page (i.e. neither the TCP, or HTTP monitor are showing you any TLS results in the Synthetics Kibana app)?

djkprojects commented 3 months ago

Hi @djkprojects thanks for the extra info. I'm setting up an 8.11.3 to try this out too.

Having set up both an TCP and HTTP monitor (to TLS enabled enabled endpoints), do you see nothing in the Synthetics > TLS Certificates page (i.e. neither the TCP, or HTTP monitor are showing you any TLS results in the Synthetics Kibana app)?

Hi Paul, that’s correct. Both TCP and HTTP monitors work fine and the synthetics-* indices do contain TLS fields with cert details but the TLS Certificates section shows nothing.

Also, if that helps, before we were using heartbeat directly (with YAML configuration) and the Uptime -> TLS certificates section worked fine. As we don't use it now the Uptime section is not showing on the menu. Mentioning it in case there is some dependency.

axrayn commented 3 months ago

Hi @djkprojects thanks for the extra info. I'm setting up an 8.11.3 to try this out too. Having set up both an TCP and HTTP monitor (to TLS enabled enabled endpoints), do you see nothing in the Synthetics > TLS Certificates page (i.e. neither the TCP, or HTTP monitor are showing you any TLS results in the Synthetics Kibana app)?

Hi Paul, that’s correct. Both TCP and HTTP monitors work fine and the synthetics-* indices do contain TLS fields with cert details but the TLS Certificates section shows nothing.

Also, if that helps, before we were using heartbeat directly (with YAML configuration) and the Uptime -> TLS certificates section worked fine. As we don't use it now the Uptime section is not showing on the menu. Mentioning it in case there is some dependency.

Hi @djkprojects,

We were getting caught out by this issue as well, it ended up being an issue with Kibana relating to the fields being used for filtering the TLS Certificates page that has now been fixed in 8.13 https://github.com/elastic/kibana/issues/178334

We were able to workaround this at the time (we couldn't upgrade at the time) by enabling status alerts on the monitor itself (even without any connector attached to the alerts)

paulb-elastic commented 3 months ago

@axrayn thanks for pointing out that Kibana issue preventing it being shown in older versions - that's a great point.

Also to mention to you both (cc @djkprojects), to include the SSL/TLS handshake for the TCP monitor, ensure you enable the corresponding setting.

I don't know if you have defined your TCP monitor configurations via the Synthetics Kibana app (the UI), or Project Monitors, but:

If a Project Monitor, ensure you are defining the ssl setting (much like what is needed in Heartbeat), as defined here
If you have configured them through the UI, then under Advanced options, ensure Use custom TLS configuration is switched on under TLS options

elastic / beats

[Heartbeat] Check certificates for Browser based Synthetic monitors (step 1 main URL) #22326