[RFC] Dynamic content creation within OSD

ruanyl commented 3 weeks ago

Background

Today, there is a growing demand for displaying various pages with customizable dynamic content within OpenSearch Dashboards (OSD). Examples include new application homepages and use case overview pages, where content is dynamically provided by plugins at runtime. This proposal aims to develop a generic solution for rendering pages with dynamic content in OSD.

Requirements

The framework should provide an interface for plugins to dynamically create and manage pages.
A page contains multiple sections
1. Section could be a dashboard container which accepts embeddable and display them in grid layout
2. Section could be a customized embeddable container which supports to display embeddable contents in various layouts
3. Section could simply be a react component that renders the contents as its children by different means
The page contents should be fully dynamic, allowing a page to aggregate and display content from multiple plugins.
The page should support embedding visualizations by specifying a visualization id
The page should support embedding dashboards by specifying a dashboard id
The page should support rendering arbitrary contents, such as React components, to provide maximum flexibility in displaying diverse types of content.

Page Structure

A page is organized into multiple sections, and each section could contain multiple contents. The entire page serves as a container for multiple sections. Each section is a full-width block within the page, designed to hold various types of content in different layouts.

Key features

Multiple Sections: The page is divided into distinct sections, each capable of containing multiple content blocks.
Flexible Layouts: Different sections can have different layout configurations, such as grid layouts or customized arrangements.
Dynamic Content: Each section can dynamically display content, allowing for displaying contents from various plugins.

Proposed Architecture

The following diagram illustrates the system allows various plugins to create and manage pages composed of multiple sections, each containing dynamic content. The goal is to provide a flexible and extensible framework that supports different content layouts and types.

Components

Plugins: Plugins are responsible for creating pages, adding sections, and populating sections with content. Multiple plugins can contribute to different parts of the page structure.

Page: A container that holds multiple sections. A page is created by a plugin and serves as the main structure for organizing content.

Section: Each section acts as a container for different types of content. Sections can be added to the page by plugins and are designed to support various layouts and content types.

Content: Individual pieces of content embedded within sections. Content can be visualizations, dashboards, custom embeddable components, or React components. Different plugins add content to sections.

Example Workflow

Page Creation:
1. A plugin creates a new page within OpenSearch Dashboards.
Section Addition:
1. Plugins add various sections (grid layout, dashboard container, custom layout, React component section) to the page as needed.
Content Addition:
1. Plugins add content blocks to the respective sections:
  1. Multiple content blocks to a grid layout section.
  2. A dashboard to a dashboard container section.
  3. Various content blocks to a custom layout section.
  4. Custom embeddable to an embeddable container.

Design details

A new plugin will be introduced for OSD that provides a content management system allowing dynamic creation and rendering of pages with various embeddable content. The following document outlines the design details includes the interfaces of the key components.

Components and Interfaces

Plugin Setup Interface

interface PluginSetup {
    registerPage: (pageConfig: PageConfig) => Page
}

interface PageConfig {
  id: string;
  title?: string;
  description?: string;
  sections?: Section[];
}

A plugin calls the registerPage method with an unique page id to create a new Page instance at the plugin setup phase. The page sections can be defined when registering a page.

There could be different types of page section, for example, a section could be a dashboard embeddable container which can hold contents, for example, visualizations, a dashboard, or any arbitrary embeddables. This type of section leverages the existing embeddable container system:

 type section = {
      kind: 'dashboard';
      id: string;
      order: number;
      title?: string;
      description?: string;
 }

You could use custom section with a render function defined which you can customize how you want to render the contents of this section:

type Section = {
      kind: 'custom';
      id: string;
      order: number;
      title?: string;
      description?: string;
      render: (contents: Content[]) => JSX.Element;
}

In a nutshell, the Section is a lightweight concept which is basically a container that defines how should the contents be displayed on the page. Thus, we can easily define new types of section which serves different purpose.

Plugin Start Interface

interface PluginStart {
    registerContentProvider: (provider: ContentProvider) => void;
    renderPage: (id: string) => React.JSX.Element
}

interface ContentProvider {
  id: string;
  getContent: () => Content;
  getTargetArea: () => string;
}

At plugin start phase, other plugins can call registerContentProvider with a given provider. The provide defines what content should be render, and which section of a page should the content be added to.

The getTargetArea: () => string; should returns a string in the format {pageId}/{sectionId} to declare the target page and section.

The getContent: () => Content; should returns the content to be render

The content could be a visualization with an input that defines the visualization id, the visualization id could be static or dynamically provided

type Content = {
      kind: 'visualization';
      id: string;
      order: number;
      input: {kind: 'static', id: string} | {kind: 'dynamic', get: () => Promise<string>};
}

Similarly, the content could be a dashboard with an input that defines the dashboard id, the dashboard id could be static or dynamically provided

type Content = {
      kind: 'dashboard';
      id: string;
      order: number;
      input: {kind: 'static', id: string} | {kind: 'dynamic', get: () => Promise<string>};
}

A content could be any renderable component, using custom content to define a render function that returns arbitrary React element.

type Content = {
      kind: 'custom';
      id: string;
      order: number;
      render: () => JSX.Element;
}

A renderPage method is exposed so that other plugins can call it to render the page and mount it, for example, homepage plugin may want to mount the homepage to a specific route /app/home.

  <Route exact path="/app/home">
    {contentManagement.renderPage('home')}
  </Route>

Summary

A new plugin contentManagement will be introduced to OSD to provide a generic and flexible framework for dynamically creating and managing pages with customizable content. By leveraging this framework, plugins can easily build rich, interactive user interfaces that integrate various types of content, including visualizations and dashboards.

Future Enhancements

Enhanced Layout Options: Additional layout options for sections could be introduced to provide more customization.
User Customization: Introducing user-specific customizations and personalization features could enhance the user experience.

Open questions

There is a requirement that user can select an existing dashboard and add it to a page, the page can display a dashboard with a given dashboard id, but how to store the user selected dashboard is still open to discuss.

wanglam commented 3 weeks ago

Shall we support call page.createSection without section id and return the created section? The createSection method will automatic generate a unique section id, then we don't need to check if the section id is unique. It will be more convenient for creating new section.

ruanyl commented 3 weeks ago

Shall we support call page.createSection without section id and return the created section? The createSection method will automatic generate a unique section id, then we don't need to check if the section id is unique. It will be more convenient for creating new section.

If multiple plugins needs add to content to the same section, they will need a way to find that specific section, in such case, I think id is required.

ashwin-pc commented 2 weeks ago

Why do we need getSections and getSections$ both? Also similarly why do we need getContents and getContents$ both?

ashwin-pc commented 2 weeks ago

What is the goal of this feature? What do we want extensible? Just content?Sections? or both? From the RFC its not clear to me how modular the content management plugin is. Right now besides the ability to register, the page and its related sections and content are very tightly coupled.

e.g. for the home page based on your PR, the underlying plugin needs to know that the get_started and some_dashboard sections exist. And if they are tomorrow for any reason changed (Since they have no obligation to be the same), the page will no longer render as expected.

export const initHome = (page: Page, core: CoreStart) => {
  page.addContent('get_started', { ... });
  page.addContent('some_dashboard', { ... });
}

A better approach might be to Introduce a system where plugins register content providers:

   interface ContentProvider {
     getContent(): Content;
     getTargetArea(): string;
     getPriority?(): number;
   }

   class ContentManagementService {
     registerContentProvider(provider: ContentProvider) {
       // Logic to add the provider's content to the appropriate area
     }
   }

Define standard areas in pages where plugins can contribute:

enum ContentAreas {
  HOME_GET_STARTED = 'home.getStarted',
  HOME_RECENT_WORK = 'home.recentWork',
  DASHBOARD_SIDEBAR = 'dashboard.sidebar',
  // ...
}

The issue right now is that Pages and sections are created explicitly, and content is added directly to sections. moving to a service that registers content independent of the page decouples plugins from page structure. Plugins don't need to know about specific section IDs or page structure.

ruanyl commented 2 weeks ago

Why do we need getSections and getSections$ both? Also similarly why do we need getContents and getContents$ both?

Right, getSections should not be exposed, updated

ruanyl commented 2 weeks ago

What is the goal of this feature? What do we want extensible? Just content?Sections? or both? From the RFC its not clear to me how modular the content management plugin is.

Multiple plugins can contribute sections to the same page, multiple plugins can contribute contents to the same section on a page. The short term goal of this feature is to make it possible to create pages for application homepage, use case overview pages which page contents are provided by different plugins.

Right now besides the ability to register, the page and its related sections and content are very tightly coupled.

A page is normally created for specific purpose, like homepage. The sections and contents on the page are normally predictable. That's why I initially designed it in this way as this looks like the most straight forward solution. But I think you have a good point to decouple the page, sections and contents. And I'm trying to understand your idea of ContentProvider, and how could it solve the problem. I'll reach you later on this.

ruanyl commented 2 weeks ago

@ashwin-pc Thank you for your suggestion on content provider, I've updated the design based on the discussions we had :)

ashwin-pc commented 2 weeks ago

Nice! I like this. Just 2 questions:

registerContentProvider: why is this in the start phase and not the setup phase? Seems like a setup thing
ContentAreas: how can the available content areas be fixed if registerPage can register new content areas?

ruanyl commented 2 weeks ago

registerContentProvider: why is this in the start phase and not the setup phase? Seems like a setup thing

At setup phase, the page and the sections will be registered, so at start phase, when register contents, the page and sections can be found. But essential, we could make it so that when registering contents, it doesn't rely on the existence of page and sections.

ContentAreas: how can the available content areas be fixed if registerPage can register new content areas?

Sections of a page are dynamic, and also different pages can have different sections. A plugin could register a page to have fix sections(content areas), but I don't tend to restrict that(fixed the page sections) at the framework level, it's up to the content management plugin consumer to define whether its page is fixed or not.

ruanyl commented 6 days ago

Initial implementation has been merged, closed by https://github.com/opensearch-project/OpenSearch-Dashboards/pull/7201

opensearch-project / OpenSearch-Dashboards