Add a containerized mode to the ECM service

apache / linkis

Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.

Apache License 2.0

3.32k stars 1.17k forks source link

What is the purpose of the change

Add a containerized mode to the ECM service, which allows assigning specific IPs and ports for communication with the outside world to particular engines in this mode. For instance, a Spark engine requires at least two ports: spark.driver.port and spark.driver.blockManager.port.

Related issues/PRs

Related issues: #5199

Checklist

[x] I have read the Contributing Guidelines on pull requests.
[x] I have explained the need for this PR and the problem it solves
[x] I have explained the changes or the new features added to this PR
[ ] I have added tests corresponding to this change
[ ] I have updated the documentation to reflect this change
[x] I have verified that this change is backward compatible (If not, please discuss on the Linkis mailing list first)
[ ] If this is a code change: I have written unit tests to fully verify the new behavior.

Why did my integration test fail when tested with linkis-cli? How can I check the failure logs? I have executed the integration test in my local environment and it passed. The specific execution command is as follows:

mvn install -Pdocker -Dmysql.connector.scope=compile -Dmaven.javadoc.skip=true -Dmaven.test.skip=true -Dlinkis.build.ldh=true
sh ./linkis-dist/helm/scripts/create-kind-cluster.sh
sh ./linkis-dist/helm/scripts/install-mysql.sh false
sh ./linkis-dist/helm/scripts/install-ldh.sh true
sh ./linkis-dist/helm/scripts/install-charts-with-ldh.sh linkis linkis-demo true
sh ./linkis-dist/helm/scripts/remote-proxy.sh start
sh ./linkis-dist/helm/scripts/login-pod.sh mg-gatewaysh
sh /opt/linkis/bin/linkis-cli -engineType shell-1 -codeType shell -code "pwd"

apache / linkis