Add date created, date modified, other metadata to Saved Objects

[cjcenizal]

Problem

Right now there's very little metadata for differentiating multiple saved objects from one another. Some metadata which would be useful would be:

• [x] ID (when we have one that's different from Name) https://github.com/elastic/kibana/pull/9087
• [x] Date created https://github.com/elastic/kibana/issues/9376
• [x] Date last modified https://github.com/elastic/kibana/issues/9377
• [x] Date last viewed https://github.com/elastic/kibana/issues/125795
• [ ] Package name (when installed by EPM)
• [ ] Managed by (Ingest manager, Beats, etc.)
• Owner (dependent upon some kind of identity system)

The lack of this metadata relates significant UI/UX improvements, e.g. adding overview tables for Dashboard and Visualize (https://github.com/elastic/kibana/pull/9605), and improved management of saved objects in Management (https://github.com/elastic/kibana/issues/125795).

Solution

Because this kind of metadata can apply to any type of document, I think this should ultimately be the role of some kind of future object model that automatically tracks this metadata (per @spalger's idea).

If we implement any kind of solution in the interim, we should keep this in mind and have a plan for migrating to this kind of system later on.

[uboness]

Owner (dependent upon some kind of identity system)

can you elaborate on that?

[cjcenizal]

Re convo w/ @uboness, instead of having owner metadata live directly on the object:

We should expose an extension point which will enable plugins to attach additional metadata attributes to saved objects. Plugins can support object level security by adding "owner" with this extension point. But what are those fields will be highly depend on the OLS design/architecture.

[alexfrancoeur]

@cjcenizal @uboness anecdotally, have you heard requests in the past for descriptive metadata around a visualization or dashboard? I could imagine an option when saving these to provide a description of what that saved objects purpose is. We could then provide that description in the list view and possibly on a tooltip within a dashboard. @acchen97 was specifically looking to provide additional information in the logstash module visualizations and I could see beats taking advantage of this as well. I can't find any issues for this request but think it's related to this meta issue.

[alexfrancoeur]

@cjcenizal monitoring actually provides this type of feature out of the box for their visualizations. This could be presented in many ways but it'd be a great addition for the user to provide additional context to a saved object.

[stacey-gammon]

Removing sharing label as I don't think this falls into our team, although I would love to see it implemented and wouldn't mind lending a hand if I ever find time.

@tylersmalley - does the new saved object API store the date created field?

[tylersmalley]

@Stacey-Gammon we already have updated_at, but lack created_at.

[alexfrancoeur]

Where do we stand on this issue? The request comes up rather frequently to be able to sort your saved objects based on some of these timestamps

[ynux]

It would be extremely helpful to get usage statistics. For one, we could drop visualizations that are never looked at. Also, users coming from tools such as Power BI or QlikView expect such information.

[TBretherton]

This would also help me out a lot. The date last viewed and also a number on a per object basis seeing how many other objects make use of it (e.g. how many times a visualization is used in a dashboard) would be ideal for when we want to delete old, unused objects.

[mostlyjason]

Added checkboxes for packages and managed by for Ingest Manager. This will help organize assets created by our integration packages, and help support read only mode https://github.com/elastic/kibana/issues/70461

[mostlyjason]

@dakrone PH mentioned that you are working on a metadata model for all of ES. Would it make sense to align with Kibana as well?

[dakrone]

@mostlyjason yes, I hope to have some ideas to share tomorrow

[sorenlouv]

Author info in saved objects I don't see this mentioned (maybe I missed it) but it would be AMAZING if saved objects also stored information about the author (created_by).

Use case I'm currently browsing the visualisations on a kibana cluster with 1000s of users. Everyone has created visualisations for their own needs. Spaces are used to separate stuff but there are still 1000s of visualisations per space making it impossible to browse the list. How do I find the visualisations I created? If saved objects stored created_by metadata, the Visualize app could take advantage of this and provide controls to filter the list of visualisations.

[cjcenizal]

Sort of related to "Add Data field descriptions" (https://github.com/elastic/kibana/issues/89726), in the sense that both are related to metadata.

[sophiec20]

+1 created_by and date_last_viewed and some kind of indicator of how much it has been recently used (in last year, say).

Use case I'm currently browsing the visualisations on a kibana cluster with 100s of users. Everyone has created visualisations for their own needs. Spaces are used to separate stuff but there are still 100s of visualisations per space. Over time, the mappings of the source data have changed through natural evolution. Many visualisations were saved during exploratory investigations and were only ever meant to be temporary. Several were created by users who have since left.

Many of the older visualisation are never likely to be used again and probably don't even work any more.

As a space owner, I'd like to be able to rationalise the visualisations and dashboards. I'd like to be able to wipe (more than half of) them so that it is easier to get to relevant information, without having to open each, one by one.

[alexfrancoeur]

cc: @elastic-jb as we begin to think through content management and governance ^^

[lukeelmers]

Author info in saved objects I don't see this mentioned (maybe I missed it) but it would be AMAZING if saved objects also stored information about the author (created_by).

updated_by and updated_at were also recently requested by @spong and @FrankHassanabad in https://github.com/elastic/kibana/issues/109189