[Feature][Task Plugin] Add Aliyun ADB Spark Task Plugin

dcoliversun commented 2 months ago

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description

Aliyun ADB Spark is a big data solution that separates data computing from data storage, which meets the requirements for online interactive search, stream processing, batch processing, and machine learning.
This feature aims to provide the Apache DolphinScheduler community users with a built-in plugin, which is also provided by Apache Airflow. We hope that this plugin will facilitate users in managing Aliyun ADB Spark applications more conveniently.

Use case

Create Datasource

Click Datasource -> Create Datasource -> ADBSPARK to create a datasource.
Fill in Datasource Name, Aliyun Access Key Id, Aliyun Access Key Secret, Aliyun Region Id and click Confirm.

Create Tasks

Click Porject -> Workflow Definition -> Create Workflow and drag the ADBSPARK task to the canvas.
Fill in the task parameters and click Confirm to create the task node.

Task Parameters

Please refer to DolphinScheduler Task Parameters Appendix Default Task Parameters section for default parameters.

Parameters	Description
Datasource types	The type of datasource the task uses, should be `ADBSPARK`.
Datasource instances	The instance of `ADBSPARK` datasource.
Aliyun ADB Cluster ID	The ID of the AnalyticDB for MySQL Data Lakehouse Edition (V3.0) cluster.
Aliyun ADB Resource Group	The name of the job resource group in AnalyticDB for MySQL Data Lakehouse Edition (V3.0) cluster.
Aliyun ADB App Type	The type of the application. Valid values: Batch(default), SQL.
Aliyun ADB Spark Data	The data of the application template. For information about the application template configuration, see Spark application configuration guide.

Related issues

No response

Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.