apache / amoro

Apache Amoro (incubating) is a Lakehouse management system built on open data lake formats.
https://amoro.apache.org/
Apache License 2.0
870 stars 289 forks source link

[Improvement]: Flink optimizer container should be summitted in detached mode when using per-job mode #3080

Open luoyell opened 3 months ago

luoyell commented 3 months ago

Search before asking

What would you like to be improved?

Run flink optimizer tasks in per-job mode, if the submitted client exits unexpedtedly while the task is running, then the task fails to run due to task manger exceptions, the job manager will hang up and cannot exit

How should we improve?

Add -d to the method of FlinkOptimizerContainer#buildOptimizerStartupArgsString to make the task run in detached mode

Are you willing to submit PR?

Subtasks

No response

Code of Conduct

czy006 commented 3 months ago

@luoyell by the way, flink per-job is deprecated in 1.17.if add -d maybe we need to know job what status in flink