[Uptime] Hide Synthetics Service Setup and Installation behind a feature flag

[awahab07]

Summary

Part of Meta Issue: https://github.com/elastic/uptime/issues/400

Add the config entries for synthetics service

xpack.uptime.unsafe.service.enabled: true
xpack.uptime.unsafe.service.password: test
xpack.uptime.unsafe.service.manifestUrl: https://test.com
xpack.uptime.unsafe.service.username: test
xpack.uptime.unsafe.service.hosts: ["test.com"]

xpack.uptime.unsafe.service.enabled Will enable the feature flag when true, and setup of the task manager, registration of saved objects to store monitor management monitor, API key generation to pass it to service and installation of fleet package for index templates will start on Kibana startup.

How To Test

• Paste the above configuration in your local <kibana-directory>/config/kibana.dev.yml and restart kibana

• When xpack.uptime.unsafe.service.enabled is true:

  • Make sure you receive the setup and installation info logs in Kibana logs (terminal when on local) e.g. You get

    [INFO ][plugins.uptime] Installed synthetics index templates

  • And/or info message that the task scheduler is registered

    [INFO ][plugins.uptime] Task UPTIME:SyntheticsService:sync-task scheduled. 

• When xpack.uptime.unsafe.service.enabled is false:

  • Make sure you don't get the above info messages
  • Make sure no errors related to uptime plugin are logged

[awahab07]

Resolved in https://github.com/elastic/kibana/pull/119446

[lucasfcosta]

I have just started post-FF testing this feature and indeed I do get the messages you mentioned in the issue when I enabled the configuration you've pasted.

• When using the following configuration, I do get the expected messages, as you can see in the screenshot below.

  xpack.uptime.unsafe.service.enabled: true
  xpack.uptime.unsafe.service.password: test
  xpack.uptime.unsafe.service.manifestUrl: https://test.com
  xpack.uptime.unsafe.service.username: test
  xpack.uptime.unsafe.service.hosts: ["test.com"]

  

• When I turn xpack.uptime.unsafe.service.enabled to false, those messages do not appear.

I'd also like to test this with the actual service running alongside Kibana before moving it into Done done, so I wonder whether I could borrow 30 minutes from someone to inform me how I could better test that. Would anyone volunteer?

[justinkambic]

FWIW I have also tested the basic feature and seen the feature flag remove the logging as expected.

[lucasfcosta]

As I was post-FF testing this, I've ran into a couple of problems. I'm not totally sure whether I've done anything incorrectly for them to appear or if they're actual problems we should look into, so I'll report them here.

Missing indexes

I have found the message in the screenshot below in the Uptime > Monitors page after setting a monitor through the monitor management UI.

That message only went away once I've ran a container with Heartbeat and setup the indexes using heartbeat setup.

To reproduce, my exact steps were:

1. Create a new cloud deployment in the LA region so that we can edit user-facing configs
2. Once the deployment is created, add these K/V pairs to the configurations for Kibana in that cloud deployment:

   xpack.uptime.unsafe.service.enabled: true
   xpack.uptime.unsafe.service.manifestUrl: [MANIFEST REDACTED]
   xpack.uptime.unsafe.service.username: example
   xpack.uptime.unsafe.service.password: [PASSWORD REDACTED]
   xpack.uptime.unsafe.service.hosts: [my_deployment_url]
   xpack.uptime.ui.unsafe.monitorManagement.enabled: true

3. Log into Kibana and create a monitor in the monitor management UI
4. Access the Uptime > Monitors

Then, to fix:

1. Run a Heartbeat container using your cloud deployment credentials so that it connects to your cloud deployment
2. Run bash within the container so that you can execute heartbeat setup
3. Visit Uptime > Monitors again and see the message is not there anymore

Monitors in the service not running

I suspect that in this case it's me doing something wrong, but I couldn't get monitors to run in the service using the monitor management UI.

How to reproduce:

1. Create a new cloud deployment in the LA region so that we can edit user-facing configs
2. Once the deployment is created, add these K/V pairs to the configurations for Kibana in that cloud deployment:

   xpack.uptime.unsafe.service.enabled: true
   xpack.uptime.unsafe.service.manifestUrl: [MANIFEST REDACTED]
   xpack.uptime.unsafe.service.username: example
   xpack.uptime.unsafe.service.password: [PASSWORD REDACTED]
   xpack.uptime.unsafe.service.hosts: [my_deployment_url]
   xpack.uptime.ui.unsafe.monitorManagement.enabled: true

3. Log into Kibana and create a monitor in the monitor management UI. I used the following configuration:
4. Access the Uptime > Monitors and keep refreshing the page. You'll see the monitor doesn't run. I've also tried the same for browser monitors with no luck

[dominiqueclarke]

@lucasfcosta To address the two concerns

Missing indices

This is currently expected until https://github.com/elastic/kibana/issues/121402. Right now, the service indexes to heartbeat-* instead of synthetics-*. Because of this, you must be run heartbeat setup against the es cluster beforehand. This is a current known limitation and not the tech-preview-ready expectation.

Service not working

There is a handful of reasons this could be happening. It may be good to check your Kibana logs and see if it reports errors. I don't believe testing the progression of the Synthetics service e2e should be in scope for this ticket, as it is an in-progress feature.

[lucasfcosta]

@dominiqueclarke makes sense, thanks for the explanation.

Considering that you've mentioned the Synthetics service tests should be out of scope for this ticket I'll then move this issue to "done done" given I've already checked the logs which @awahab07 mentioned above when explaining how to test it.