apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
https://dolphinscheduler.apache.org/
Apache License 2.0
12.8k stars 4.59k forks source link

[Feature][Data Quality] Data Quality Job Support Multiple Rules #13524

Open qingwli opened 1 year ago

qingwli commented 1 year ago

Search before asking

Description

For now, data quality only supports a single rule for one job, and for our real scenes to be used, one job needs to check a lot of fields. So we want to support multiple rules for one data quality job.

Use case

We will change the data quality task definition page to a table and then can add a lot of rules and crud, below are the demo design:

定义任务

新建规则 组合页面

And we will change the data quality result page to support a result with multiple rules, below is the demo design:

任务结果 任务明细

Related issues

Enhance https://github.com/apache/dolphinscheduler/issues/4283

Are you willing to submit a PR?

Code of Conduct

github-actions[bot] commented 1 year ago

Search before asking

Description

For now, data quality only supports a single rule for one job, and for our real scenes to be used, one job needs to check a lot of fields. So we want to support multiple rules for one data quality job.

Use case

We will change the data quality task definition page to a table and then can add a lot of rules and crud, below are the demo design:

定义任务

新建规则 组合页面

And we will change the data quality result page to support a result with multiple rules, below is the demo design:

任务结果 任务明细

Related issues

Enhance https://github.com/apache/dolphinscheduler/issues/4283

Are you willing to submit a PR?

Code of Conduct

github-actions[bot] commented 1 year ago

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.

zhongjiajie commented 1 year ago

Good idea, it is a sweet feature and I like it. looking forward your contribution

liujinhui1994 commented 1 year ago

@liqingwang Is there a PR for this now?

qingwli commented 1 year ago

Hi @liujinhui1994 , Sorry for the later reply, It's still working, some work things have higher priority, will take 1-2 months

github-actions[bot] commented 5 days ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.