[RoadMap][Year 2024] Community RoadMap

[zhongjiajie]

• For some topics which need further discussions, we temporarily note the owner as Community.

RoadMap 2024

• User Friendly
  • [ ] Audit log improvement design #15423
  • [ ] [TriggerPlugin] Introduce Trigger Plugin #15330
• New Important Task or Task Related
  • [ ] Dynamic Task Plugin #12526
  • [ ] Independent Release of Task Plugin
  • [ ] Support Streaming Task #11352
  • [ ] Task #11652
  • [ ] Lifecycle method to TaskPlugin #14877
  • [ ] Unify the task spi about logic task and worker task #14823
• Test Improvement
  • [ ] API Test #10411
  • [ ] UT Improvement #10573
• Data Ops
  • [ ] Git Ops Support
  • [ ] Git CICD related

[github-actions[bot]]

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.

• In order for us to understand your request as soon as possible, please provide detailed information, version or pictures.
• If you haven't received a reply for a long time, you can join our slack and send your question to channel #troubleshooting

[zhongjiajie]

Y2022Q4 roadmap in https://github.com/apache/dolphinscheduler/issues/12436

[rickchengx]

Hi, @zhongjiajie , Here are the features I'd like to implement in Y2023Q1 :)

1. [Feature] Remote Logging on more targets (Google Cloud Storage, S3, Azure Blob Storage, Etc.)

   • 13017

2. [Feature] Extending the Storage Backend for Resource Center (Google Cloud Storage, Azure Blob Storage)

   • 13481

   • 13428

[davidzollo]

if you'd like to contribute any feature to the Apache DolphinScheduler, please leave a message.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[EricGao888]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

No stale.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[EricGao888]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

no stale

[zhongjiajie]

Summary Roadmap Finish in 2023

• Cloud Native
  • [x] Remote Logging #13017
  • [x] K8S Operator https://github.com/apache/dolphinscheduler-operator
• New Important Task or Task related
  • Remote Task
  • [x] #13794
  • [x] Dynamic Worklfows Generate #14125

[gzusgw]

• For some topics which need further discussions, we temporarily note the owner as Community.

RoadMap 2024

• Cloud Native

  • [ ] K8S Executor [Feature][Executor] Add K8S Executor for task dispatching #13316

• User Friendly

  • [ ] Dynamic Task Plugin DSIP-20: Basic version of the dynamic task component (only implements a required field in a Shell component). #12526
  • [ ] Task Parameters Sharing Optimization
  • [ ] Independent Release of Task Plugin
  • [ ] Audit log improvement design [Improvement][Audit log] Audit log improvement design #15423

• New Important Task or Task Related

  • [ ] Support Streaming Task [DSIP-16][Task] Support stream task #11352

  • Remote Task

  • [ ] [Feature] DS can support task running on remote host, not just worker server. #11652

  • [ ] [TriggerPlugin] Introduce Trigger Plugin [DSIP-22][TriggerPlugin] Introduce Trigger Plugin #15330

  • [ ] Task Sharing Storage

  • [ ] Trigger Workflow by Event

  • [ ] Lifecycle method to TaskPlugin [DSIP-23][TaskPlugin] Add Lifecycle method to TaskPlugin  #14877

  • [ ] Unify the task spi about logic task and worker task [DSIP-21][TaskPlugin] Unify the task spi about logic task and worker task  #14823

• Test Improvement

  • [ ] API Test
  • [ ] UT Improvement [DSIP-10][Unit Tests] Improve DolphinScheduler unit tests #10573
  • [ ] [DSIP-22][TriggerPlugin] Introduce Trigger Plugin #15330

• Data Ops

  • [ ] Git Ops Support
  • [ ] Git CICD related

could add loop task？ https://github.com/apache/dolphinscheduler/issues/14619

[zhongjiajie]

• For some topics which need further discussions, we temporarily note the owner as Community.

RoadMap 2024

• Cloud Native

  • [ ] K8S Executor [Feature][Executor] Add K8S Executor for task dispatching #13316

• User Friendly

  • [ ] Dynamic Task Plugin DSIP-20: Basic version of the dynamic task component (only implements a required field in a Shell component). #12526
  • [ ] Task Parameters Sharing Optimization
  • [ ] Independent Release of Task Plugin
  • [ ] Audit log improvement design [Improvement][Audit log] Audit log improvement design #15423

• New Important Task or Task Related

  • [ ] Support Streaming Task [DSIP-16][Task] Support stream task #11352

  • Remote Task

  • [ ] [Feature] DS can support task running on remote host, not just worker server. #11652

  • [ ] [TriggerPlugin] Introduce Trigger Plugin [DSIP-22][TriggerPlugin] Introduce Trigger Plugin #15330

  • [ ] Task Sharing Storage

  • [ ] Trigger Workflow by Event

  • [ ] Lifecycle method to TaskPlugin [DSIP-23][TaskPlugin] Add Lifecycle method to TaskPlugin  #14877

  • [ ] Unify the task spi about logic task and worker task [DSIP-21][TaskPlugin] Unify the task spi about logic task and worker task  #14823

• Test Improvement

  • [ ] API Test
  • [ ] UT Improvement [DSIP-10][Unit Tests] Improve DolphinScheduler unit tests #10573
  • [ ] [DSIP-22][TriggerPlugin] Introduce Trigger Plugin #15330

• Data Ops

  • [ ] Git Ops Support
  • [ ] Git CICD related

could add loop task？ #14619

not sure whether is like dynamic task, so I add some comment on it, which you can see in https://github.com/apache/dolphinscheduler/issues/14619#issuecomment-1925563753

[Dyqer]

Hi, Team, How about to create a Java SDK?

[lenboo]

put it back

• Cloud Native
  • [ ] K8S Executor #13316