bytedance / bitsail

BitSail is a distributed high-performance data integration engine which supports batch, streaming and incremental scenarios. BitSail is widely used to synchronize hundreds of trillions of data every day.
https://bytedance.github.io/bitsail/
Apache License 2.0
1.63k stars 335 forks source link

[Bug][bitsail-entry] missing jobId #490

Closed pursuit-wangpz closed 1 year ago

pursuit-wangpz commented 1 year ago

What happened

bitsail将任务提交到yarn后,如何通过bitsail命令行停止任务。使用以下命令,报错找不到入参job-id。 日志: 17:10:25.801 [main] INFO com.bytedance.bitsail.client.entry.Entry - Engine flink's command: [/home/spark/bitsail-archive-0.2.0-SNAPSHOT/embedded/flink/bin/flink, stop, -t, yarn-per-job]. SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/spark/bitsail-archive-0.2.0-SNAPSHOT/embedded/flink/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/TDP/hadoop-3.3.2/share/hadoop/common/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Missing JobId

Use the help option (-h or --help) to get help on the command. 操作命令:sh -x bin/bitsail stop --engine flink --deployment-mode yarn-per-job --execution-mode stop --job-id XXXXXXX

What do you expect to happen

入参--job-id,应该是支持的

How to reproduce

BitSail run command: 'sh -x bin/bitsail stop --engine flink --deployment-mode yarn-per-job --execution-mode stop --job-id XXXXXXX'

Build Environment

No response

Execution Environment

No response

BitSail version

master

BitSail Component or Code Module

BitSail Core

Are you willing to submit PR?

Code of Conduct

garyli1019 commented 1 year ago

目前还不支持stop命令,你指的job id是一个任务每次调度都保持一样的一个值吗,不确定yarn有没有这种方式来kill

pursuit-wangpz commented 1 year ago

好的,谢谢。我指的job-id更像是yarn的applicationId这种。