[Unified o11y] What mechanisms do we have to show information in the overview page?

afgomez commented 2 years ago

We want to redesign the observability overview page. We want to show a series of graphs, etc. We have several options to do so:

Option 1: Embed a Kibana dashboard

(Not sure if this is possible or to what extend).

This approach will involve creating a Dashboard and embedding it in the overview page. The effort is mostly in three palces:

Creating the visualizations and the dashboard
Figure out how to embed it in the page.
Create embeddables for any custom visualizations that we need.

We also need to determine how this dashboard will show up within Kibana's own "Dashboard" page, and if we should allow users to modify it from there.

Pros	Tradeoffs
- Querying the data is handled by the dashboard - Personalization comes included - Custom embeddables can be reused in other places, even by our customers.	- Users might break the overview page by changing the dashboard - The advantage of this versus linking directly to a dashboard is not clear - Lens visualizations only work with data views (index patterns).

Option 2: Lens embeddables

Lens provides an Embeddable that allows consumers to insert any lens visualization anywhere within kibana. If any of the visualizations that we need are available within lens we can use that, and then make custom components for what we need.

Pros	Tradeoffs
- Plenty of graph types. - Data fetching is handled by Lens - We automatically get all improvements from Lens	- Data querying is limited to what Lens can do. - Limited to Lens' graph types. It will need hand-made elements for some stuff. - If we want to allow the user to move things around we need to reinvent dashboards. - Lens can only work with data views (index patterns)

Option 3: Synthetics' Exploratory View

The Synthetics team has created a wrapper around Lens Embeddables with predefined datasets and reports that end up generating a graph.

We can add custom data types and reports. Most of the effort will go here.

Pros	Tradeoffs
- Same as Lens - The data types and reports that we add will automatically be available in the exploratory view.	- Same as Lens

Option 4: Custom everything

What the title says. Build everything ourselves.

Pros	Tradeoffs
- Everything exactly as we need	- Not reusable.

jasonrhodes commented 2 years ago

I spoke with @stacey-gammon and she pointed me to a conversation from a while ago with @andrew-goldstein from Security Solution:

@andrew-goldstein / @stacey-gammon would you mind helping us understand what you all were trying to do here and what ultimately happened with this effort? We are trying to understand what's currently possible today re: embedding dashboards.

jasonrhodes commented 2 years ago

@jen-huang / @joshdover can you point us to any ongoing conversations about dealing with "read-only assets" in Kibana? We are exploring the idea of embedding dashboards in our curated observability UIs and I know that is going to bring in a bunch of problems with users deleting and/or editing these assets, and I know this is something on your radar. We would just like to see what is on the roadmap, if anything, re: this problem.

stacey-gammon commented 2 years ago

I think you want @lukeelmers, not Josh. IIRC, embedded dashboards were featureful enough where the security team could start using them, but I suspect the feature was de-prioritized and it just was never implemented. There is an example plugin that shows how to use the by-value dashboard option. That might be what you want given that there is no saved object to manipulate.

cc @vadimkibana in case you have any additional thoughts to add.

andrew-goldstein commented 2 years ago

I spoke with @stacey-gammon and she pointed me to a conversation from a while ago with @andrew-goldstein from Security Solution:

[Meta] Embeddable types #43879 (comment)

[Embeddable] Dashboard by reference embeddable #52680 (comment)

@andrew-goldstein / @stacey-gammon would you mind helping us understand what you all were trying to do here and what ultimately happened with this effort? We are trying to understand what's currently possible today re: embedding dashboards.

The goal described in https://github.com/elastic/kibana/issues/52680#issuecomment-564634798 :

Goal

The SIEM team aims to proffer users the ability to select from a list of pre-populated and user-generated Kibana dashboards (stored as saved objects), and render them via the embeddable dashboard component.

Ideally, the embeddable dashboard component would also allow changes made to the dashboard in edit mode, (e.g. resizing a widget) to be persisted back to the saved dashboard if the user explicitly invokes a save action.

Limitations in the current implementation

Reads

Today, all apps that wish to populate an embedded dashboard with the contents of a saved Kibana dashboard must develop their own implementation of the code that performs the IO to retrieve the saved object via the Saved Objects API, and transform it in-memory to the DashboardContainerInput, shown in the references section below.

Alternatives to the current implementation for reads might include:

Passing the saved object ID of a Kibana dashboard as a prop to the embeddable dashboard, such that the embeddable dashboard component handles the IO for retrieving the saved object, and transforming it for rendering

Deferring the IO for perstiting the saved object to the app, but providing a utility function that converts from the saved object format to the DashboardContainerInput format

Writes

When users (for example) change the layout of a dashboard in edit mode, it's not possible (today) to persist those layout changes to the saved object representation of the dashboard.

Potential enhancements to the embeddable dashboard to support writes might include:

Passing the saved object ID of a Kibana dashboard as a prop to the embeddable dashboard, and enhancing the embeddable dashboard component to provide a UI action that, when clicked, persists the new layout to the saved object

Deferring the IO for persisting the mutated layout to the app, but providing a hook or callback that will be invoked when the user mutates the dashboard, and provide a utility function that converts from the in-memory representation of the dashboard to the saved object format

is still relevant, and would enhance multiple views in the Security Solution if it could be realized.

This comment has links to the issues that were still blockers the last time the team discussed this topic.

cc @paulewing @monina-n

afgomez commented 2 years ago

@stacey-gammon thanks for your response.

That might be what you want given that there is no saved object to manipulate.

For now this is fine and we can hard-code the values, but eventually it is interesting for us to use a saved object.

lukeelmers commented 2 years ago

can you point us to any ongoing conversations about dealing with "read-only assets" in Kibana?

@jasonrhodes Here is the relevant GH issue: https://github.com/elastic/kibana/issues/70461

rayafratkina commented 2 years ago

cc @sixstringcode @ghudgins @VijayDoshi as this is related to the composability of apps & flow from solutions we were discussing recently

elastic / kibana