[Bug] [dolphinscheduler-worker] Unable to stop tasks in Seatunnel cluster

[cqutwangyu]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

我通过DS定时调度，使用Cluster模式启动Seatunnel任务。 然后通过DS页面中“工作流实例”->“操作”->“停止”按钮停止这个Seatunnel任务。

结果是，杀死了SeatunnelClient。 但事实上SeatunnelClient在Cluster模式下，只是讲任务提交到SeatunnelServer，真正执行任务在SeatunnelServer。 所以，杀死SeatunnelClient，只在Local模式下有用。 在Cluster模式下，需要调用Seatunnel提供的rest api -> "/hazelcast/rest/maps/stop-job"。

我本想自己实现在结束Seatunnel任务的同时调用这个接口，但我没有看到dolphinscheduler-task-seatunnel或dolphinscheduler-worker支持这样的扩展。

What you expected to happen

pass

How to reproduce

pass

Anything else

pass

Version

dev

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[github-actions[bot]]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

I use DS scheduled scheduling and use Cluster mode to start the Seatunnel task. Then stop this Seatunnel task through the "Workflow Instance" -> "Operation" -> "Stop" button on the DS page.

As a result, the SeatunnelClient is killed. But in fact, SeatunnelClient in Cluster mode only submits tasks to SeatunnelServer, and actually executes tasks on SeatunnelServer. Therefore, killing SeatunnelClient is only useful in Local mode. In Cluster mode, you need to call the rest api provided by Seatunnel -> "/hazelcast/rest/maps/stop-job".

I originally wanted to implement calling this interface while ending the Seatunnel task, but I did not see that dolphinscheduler-task-seatunnel or dolphinscheduler-worker supports such an extension.

What you expected to happen

pass

How to reproduce

pass

Anything else

pass

Version

dev

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[JohnZp]

Bro, did you solve it? I also encountered the same problem; In the latest version I am using now, temporarily suspending the task flow will cause the task flow to get stuck and stop the task. The task does not really stop; My idea is to collect the jobid through logs, add a command line to the cancel method, and process the cancel method through SEATUNNLE_HOME/seatunnel.sh --cancel-job/--savepoint.

[cqutwangyu]

Bro, did you solve it? I also encountered the same problem; In the latest version I am using now, temporarily suspending the task flow will cause the task flow to get stuck and stop the task. The task does not really stop; My idea is to collect the jobid through logs, add a command line to the cancel method, and process the cancel method through SEATUNNLE_HOME/seatunnel.sh --cancel-job/--savepoint.

Good idea, maybe you could try contributing source code.

[JohnZp]

Bro, did you solve it? I also encountered the same problem; In the latest version I am using now, temporarily suspending the task flow will cause the task flow to get stuck and stop the task. The task does not really stop; My idea is to collect the jobid through logs, add a command line to the cancel method, and process the cancel method through SEATUNNLE_HOME/seatunnel.sh --cancel-job/--savepoint.

Good idea, maybe you could try contributing source code.

I made the Seatunnel Stream task a separate plugin, similar to Flink Stream, doing separate tasks to start and stop and savepoint, but the customization is too strong, I don't know how to push it to the community

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

[davidzollo]

Some user told me that task can't stop and he provided the following picture, SeaTunnel version is 2.3.5 or 2.3.6

but we don‘t find unusual cause when checking the source code. Usually SeaTunnel process will be killed in cluster mode once you click stop button. If any user meet the same problems, please provide some more messages such as SeaTunnel process screenshot in detail

By the way, if you have solved the problem, please also leave a message, thx

[ziya05]

Some user told me that task can't stop and he provided the following picture, SeaTunnel version is 2.3.5 or 2.3.6

but we don‘t find unusual cause when checking the source code. Usually SeaTunnel process will be killed in cluster mode once you click stop button. If any user meet the same problems, please provide some more messages such as SeaTunnel process screenshot in detail

By the way, if you have solved the problem, please also leave a message, thx

I found if I create a seatunnel task which read data from kafka and save data to database, while set the job.mode to "STREAM", then stop the task after run it, the data will be update in database, and I can find the job from seatunnel REST Api.

By the way I hope dolphinscheduler will support seatunnel stream task, then I can manager the task in "realtime task".

[pzc-hhh]

Bro, did you solve it? I also encountered the same problem; In the latest version I am using now, temporarily suspending the task flow will cause the task flow to get stuck and stop the task. The task does not really stop; My idea is to collect the jobid through logs, add a command line to the cancel method, and process the cancel method through SEATUNNLE_HOME/seatunnel.sh --cancel-job/--savepoint.

Good idea, maybe you could try contributing source code.

I made the Seatunnel Stream task a separate plugin, similar to Flink Stream, doing separate tasks to start and stop and savepoint, but the customization is too strong, I don't know how to push it to the community

Can you show me the code and how you modified the method for canceling tasks in Seatunnel? Thank you.