[Uptime] Migrate from custom telemetry sender to the new Core-provided one

afharo commented 2 years ago

Hi folks!

I'm creating this issue to track the effort for migrating the current custom telemetry sender in x-pack/plugins/synthetics/server/legacy_uptime/lib/telemetry/sender.ts to the new Core APIs to send telemetry events: core.analytics.

The full documentation can be found in: https://docs.elastic.dev/telemetry/collection/event-based-telemetry

Essentially, during the setup phase of your plugin, you need to register the event and the structure that you're planning to send via:

export class MyPlugin extends Plugin {
...
  public setup(core, pluginDependencies) {
    core.analytics.registerEventType({
      eventType: 'my_uptime_event_name',
      schema: {...}, // Declaration of the structure of the event
    });
  }
...
}

Then, anytime you need to ship any events, you can call core.analytics.reportEvent('my_uptime_event_name', {...contentOfTheEvent});. The API reportEvent is available in the setup and start contracts.

Some relevant documentation about the APIs can be found in the client's package readme: https://github.com/elastic/kibana/blob/main/packages/analytics/client/README.md#reporting-events:

Added benefits

You can send telemetry directly from the browser when the events are related to UI actions.
This API has the concept of Context Providers (refer to https://github.com/elastic/kibana/blob/main/packages/analytics/client/README.md#enriching-events for how to implement them). It allows enriching all the events with relevant data like the already existing license, cloud deployment ID, cluster_uuid, version, server's status, and many more, so you don't need to provide it in your own events.

Changes to your current implementation

Looking at your implementation, you would call events.forEach((event) => core.analytics.reportEvent(channel, event)) inside your queueTelemetryEvents method, and you can delete all the rest 😇

Remember that, during the setup phase, you'll need to declare all the events you report MonitorUpdateTelemetryChannelEvents.

Testing

There are some Jest mocks you can use for unit tests. They are exported alongside the core mocks.

But we also offer some FTR helpers to make developing FTR tests much easier. You can find them in test/analytics in the Kibana repo.

Indexer considerations

⚠️ You may need to create a new indexer for this new implementation considering these 2 changes:

The structure of the events with the new API looks like this:

{
  "event_type": "my_uptime_event_name",
  "timestamp": "2022-08-03T00:00:00.000Z", // <- automatically populated the moment you call `reportEvent`
  "context": {...}, // All the info coming from the Context Providers
  "properties": {...} // The event data provided as the second parameter in the call to `reportEvent`.
}

The target channels that the core API sends this data to are kibana-server and kibana-browser, depending on where the event was generated (on the server or the browser, respectively).

Relates to https://github.com/elastic/kibana/issues/137845

elasticmachine commented 2 years ago

Pinging @elastic/uptime (Team:uptime)

paulb-elastic commented 2 years ago

@afharo with https://github.com/elastic/kibana/pull/137828 fixing https://github.com/elastic/kibana/issues/137845 for Uptime and Synthetics, is this still needed?

afharo commented 2 years ago

@paulb-elastic AFAIK, the agreement was to use the custom telemetry sender temporarily until Core provided a common API to ship event-based telemetry (which is available from 8.3).

The current implementation has some fundamental known bugs that are spread through all the implementations that were copied across (from Security Solutions to Uptime, osquery (#137963) and fleet (#137964)) while waiting for the general API.

paulb-elastic commented 1 year ago

@afharo we're not intending to do any updates to Uptime. What impact would this have if we were to leave Uptime as it is?

elastic / kibana