search

opensearch-project / OpenSearch-Dashboards

📊 Open source visualization dashboards for OpenSearch.
https://opensearch.org/docs/latest/dashboards/index/
Apache License 2.0
1.67k stars 875 forks source link

[Meta][2.17] Support Multiple Data Source in OpenSearch Dashboards Plugins #7578

Open BionIT opened 2 months ago

BionIT commented 2 months ago

Context

Since 2.4.0, multiple data source feature is enabled as a core plugin(see RFC https://github.com/opensearch-project/OpenSearch-Dashboards/issues/1388 and PR https://github.com/opensearch-project/OpenSearch-Dashboards/pull/2334).

With multiple data source enabled, users of OpenSearch Dashboards are able to create data source connections and query local and remote OpenSearch clusters(see documentation https://opensearch.org/docs/latest/dashboards/management/data-sources/). Enabling multiple data source at dashboards plugin would ensure users that have been using the multiple data source feature to have a consistent experience in the dashboard, and allows users to use the same dashboard for different clusters.

This meta issue is used to track the PR/Issues for supporting multiple data source in OpenSearch Dashboards Plugins

Data source connection picker

In https://github.com/opensearch-project/OpenSearch-Dashboards/issues/5717, we introduced a picker to select data connection. This picker is within the data source plugin bundle, and when loaded, it will fetch data source connection which is a type of saved object and then populate the available data connections as single selected options in the drop down.

In order to consume this picker from a plugin, follow the steps as below:

  1. in the opensearch_dashboards.json file, add dataSource as required bundle, eg. "requiredBundles": ["dataSourceManagement"]
  2. import the component in the file where this picker is needed, eg. import { ClusterSelector } from '../../data_source_management/public'
  3. this picker exposes onSelectedDataSource which can be used to get the selected option from the picker component

Data source client

Data source plugin was released in OpenSearch Dashboards 2.4.0 as part of the multi data source feature. The plugin exposes OpenSearch client wrapper and this is the design https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/docs/multi-datasource/client_management_design.md, it can be acquired from context via the following: client: OpenSearchClient = await context.dataSource.opensearch.getClient(dataSourceId); For legacy client, we can initiate the client via the following apiCaller: LegacyAPICaller = context.dataSource.opensearch.legacy.getClient(dataSourceId).callAPI; Note that we need to pass in the data source id as argument, and this id can be acquired by passing a function to the child component by following instructions above.

PoC

PoC commit for query bench dashboards plugin to support MDS: https://github.com/opensearch-project/dashboards-query-workbench/compare/main...BionIT:dashboards-query-workbench:mdinteg

A few things to notice:

  1. to consume the client, we need to add "optionalPlugins": ["dataSource"] in opensearch_dashboards.json in the plugin repo
  2. to consume the React component to pick data source, we need to add "requiredBundles": ["dataSourceManagement"]
  3. during plugin set up, in order to register the plugin API schema/spec, we can do the following dataSource.registerCustomApiSchema(sqlPlugin)
  4. most plugins might need to depend on the data source picker, please follow to set it up, and once we acquire the selected data source id, we need to pass it to plugin const {dataSourceId} = request.query; then the datasource Id can be passed to the data source client wrapper context.dataSource.opensearch.getClient(dataSourceId)
kuzaxak commented 1 week ago

Hey, do you plan to release it soon?