search

apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
https://dolphinscheduler.apache.org/
Apache License 2.0
12.93k stars 4.64k forks source link

[FEATURE] Is it possible to add hyperlinks between workflow instance and its task instances? #1240

Closed PhenixXX closed 4 months ago

PhenixXX commented 5 years ago

For example, if every workflow instance has an independent DAG gram with hyperlinks to its task instances, and every task instance also has hyperlink to the workflow instance it belongs to, we can fast-locate the tasks and workflow when problem happens. I think it's useful when we have a complex procedure of data processing.

davidzollo commented 5 years ago

awesome，good idea!

anybody who like this feature please leave message!

PhenixXX commented 5 years ago

@dailidong Thanks a lot! I do data mining so I pay more attention to its pain points.

Since there're already tree diagrams for workflow instances group, I suggest hyperlinks start from there, this will be able to connect worflow group, workflow instances and task instances. Besides, addtional functions like "multi-selecting" and "batch-running" can be added here, since backfills or re-running are not always sequential.

As to DAG diagram for each workflow instance, I think it needs independent page where we can re-oganize dependencies inside the workflow instance( without effecting the group definition). For example, we can re-run the workflow from any node, it alwasy goes along downstream dependencies until the end or a node with multi-upstream dendencies. If we break or add certain dependency, the temporary-workflow goes and ends exactly the way we need. This, I suppose, is very useful for those long and halfway-interrupted workflow instances.

For your referrence. Thanks again.

davidzollo commented 5 years ago

good advice! would you like to implement this feature?

PhenixXX commented 5 years ago

I'd really like to, however, I'm not good at coding. All my code jobs are like data ETL, data analysis with Python. I don't know what skills are needed for developing but I'm not qualified for now, I'm afraid. Do you have any advice if I'd like to participate in this project?

davidzollo commented 5 years ago

Oh，for now , DolphinScheduler is written by java, if you have interesting in coding, you can study java, once you master the java, you can try fix some small bugs. and I want to say: besides this, you also can do some not-code work for Dolphin, for example, Write some documents, answer issue if you know, submit issues and so on. The current documents are not perfect. there are many works need us to do!

you can add my wechat: dailidong66, and tell me what do you want to do.

thx!

davidzollo commented 5 years ago

please remember some apache project committers not write code, they maybe a person who writes documents, organizes activities, or evangelist and so on