elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.48k stars 8.04k forks source link

[Epic] Create data access plugins for observability signals #161875

Open jasonrhodes opened 11 months ago

jasonrhodes commented 11 months ago

We are going to solve the problem of data access across observability plugins once and for all. :) To do this, we're going to explore this idea of creating a "tier" of data access plugins (read more about plugin tiers) that provide access to signal data. This epic specifies a plan for gradually moving to this goal, with task lists to track progress.

Level 1

In this level, each existing observability plugin creates a data access plugin which should, at a minimum, expose a single method that returns the names of the indices where signal data can be found.

### Tasks
- [ ] #161876 
- [ ] https://github.com/elastic/kibana/issues/161906

Level 2

The data access queries that are performed inside of "Tier 1" UI plugins today would be moved into the data access plugin and exposed as methods on a server and browser client, available via the data access plugin's lifecycle methods.

[Task list TBA]

Level 3

Data access plugins that mimic their UI plugin's structure should be broken into signal-based concerns, e.g. if we start in level 1 with an "APM" data access plugin, in this level we would aim to break that into a "traces" data access plugin. Application metrics would become available via the metrics data access plugin, and services would become available via an asset client.

[Task list TBA]

Plugin Tiers

To help with our runtime dependency management and avoiding unexpected circular dependency problems, we are introducing the idea of "plugin tiers" as a conventional concept. This is not currently enforced in any way, but it helps us understand how to talk about which plugins "should" depend on which others.

Screenshot 2023-07-13 at 10 23 25 AM

In this system, plugins may only depend on other plugins that are in a tier above the one they are in.

For observability, this would look something like this.

Screenshot 2023-07-13 at 10 26 31 AM

The data access tier is the new tier we are introducing with this epic. Dependency flow would look something like this, where there are a lot more dependencies between plugins than we currently have, but all of these are "safe" from a circular dependency perspective.

Screenshot 2023-07-13 at 10 29 19 AM
elasticmachine commented 11 months ago

Pinging @elastic/unified-observability (Team:Unified observability)

roshan-elastic commented 8 months ago

Closing - done

jasonrhodes commented 8 months ago

This isn't done yet, it's an ongoing epic with multiple levels described in the description.