[RFC]Integration resource Seperation into multi-part resources

[YANG-DB]

Is your feature request related to a problem?

Integration Context

Integrations are a structured based bundle of assets that are pre-build with a specific domain use-case that represents its behaviour and purpose. Integrations are located in the catalog repository and are used as a library for loading pre-build dashboards and data views for specific domains such as observability. The Dashboard-Observability plugin contains the actual framework components that present the integration template and allow uploading new integrations catalog libraries.

Integration Existing Problems

As of today, the integration framework is based on the assumption that we can create a single self-contained asset that will perform all the activities and use cases that reflect the concept of an integration.

As for any other ndjson assets, the Integration asset is limited in size due to the dashboard's restriction of 1 MB. In addition to that, the integration is in many cases represent multiple aspects of a specific resource - Nginx for example. We would like to be able to associate different use-cases based assets related to that integration but not necessarily contained within that integration or having a similar release line.

Lets review the getting-started use-case that is associated with a specific resource:

• Getting started instructions for a docker based quick start
• Getting started instructions for a collector or agent based data shipment into opensearch

This component is an example of the general notion that an integration is a super structure that my have multiple associated use cases and flow, these can all be coupled with a specific integration which is related to a specific resource (Nginx for example). In addition, the existing structure of an integration is hard coded with the domain it represents, this can become more generic using the described notion of the integration being the parent of all the implementing domains related to that specific resource.

What solution would you like?

Suggested Framework Updates

Today In the Dashboard core there is a concept of a relationships between different savedObject assets including a parent child relationship. This RFC is intended to take advantage of this feature and apply it to the integration; integration would become the parent of all the associated use-cases for that resource and in particular for the getting-started use case.

This approach will assist in multiple ways:

• allow creation of multiple-schema based domains to a single resource
• enable larger sized assets and bundle of content to be imported into the savedObject
• decouple different assets version from the parent integration thus allowing delivery
• simplify customer's choice of resources they would like to use as part of the integration
• formulate the notion of a resource use-cases catalog in a fine grained way

What alternatives have you considered? A clear and concise description of any alternative solutions or features you've considered.

Do you have any additional context?

• https://github.com/opensearch-project/opensearch-catalog/issues/165
• https://github.com/opensearch-project/opensearch-catalog/issues/144
• https://github.com/opensearch-project/dashboards-observability/issues/1929

Example of parent-child saved object relationship between different ndjson assets:

[dblock]

Catch All Triage - 1, 2, 3

[dblock]

@YANG-DB This repo should be part of an existing triage, help include it?