[Feature][ECM] ECM容器化后支持spark引擎driver端与集群之间的通信

[sjgllgh]

Search before asking

• [X] I had searched in the issues and found no similar feature requirement.

Problem Description

目前ECM容器化后，在使用spark引擎时可能会有以下问题：

1. 当spark使用client模式提交任务是，集群会和driver端有通信，涉及如下几个网络配置信息：
   • spark.driver.host
   • spark.driver.port
   • spark.driver.blockManager.port
2. 在对ECM进行容器化部署时，需要通过配置好的vnc访问，且开放端口是有限的
3. 可能会有其他大数据组件有这个问题，所以希望能够在ECM中实现这个功能，并在引擎启动时，为引擎分配可以外接通信的ip和端口

Description

No response

Use case

No response

Solutions

No response

Anything else

No response

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR!

[github-actions[bot]]

:blush: Welcome to the Apache Linkis community!!

We are glad that you are contributing by opening this issue.

Please make sure to include all the relevant context. We will be here shortly.

If you are interested in contributing to our website project, please let us know! You can check out our contributing guide on :point_right: How to Participate in Project Contribution.

Community

WeChat Assistant	WeChat Public Account

Mailing Lists

Name	Description	Subscribe	Unsubscribe	Archive
dev@linkis.apache.org	community activity information	subscribe	unsubscribe	archive

[peacewong]

Good features