search

apache / submarine

Submarine is Cloud Native Machine Learning Platform.
https://submarine.apache.org/
Apache License 2.0
696 stars 252 forks source link

how to run it on yarn #980

Open tonywang-sh opened 2 years ago

tonywang-sh commented 2 years ago

I didn't find submarine usage guide for running on yarn. Whether its ver 0.7 does support to run task on yarn? Any guide documents? Thanks.

cdmikechen commented 2 years ago

@tonywang-sh Hi~ You can try 0.6.0. Here is the document https://submarine.apache.org/docs/0.6.0/adminDocs/yarn/

tonywang-sh commented 2 years ago

From v0.6.0 yarn document, there is the cmd line, "java org.apache.submarine.client.cli.Cli job run --name tf-job-001 \ ....". Where is the 'java org.apache.submarine.client.cli.Cli' implementation? I didn't find its package with Cli interface. Thanks.

cdmikechen commented 2 years ago

@tonywang-sh It is in submarine-client, and you need to change tag to release-0.6.0

tonywang-sh commented 2 years ago

Does release v0.7 or later version support to run on yarn?

tonywang-sh commented 2 years ago

In addition, Does release-0.6.0 support hadoop 3.2.2? It reported errors when compiling the source codes of release-0.6.0 as the below. image

cdmikechen commented 2 years ago

@tonywang-sh Can you show your build command and mvn/java version? There is a profile hadoop-3.2 in pom.xml, hadoop 3.2 should be compiled if it is configured correctly.

tonywang-sh commented 2 years ago

release-0.6.0 showed the errors with hadoop 3.2 as the below. image

cdmikechen commented 2 years ago

@tonywang-sh Can you show your build command and mvn/java version? I run mvn clean install -DskipTests -Phadoop-3.2 and build succeeded.

tonywang-sh commented 2 years ago

I tried the cmd line, mvn clean install -DskipTests -Phadoop-3.2.2, but it failed. Another issue, release-0.7.0 or later versions don't have java code about yarn, how to run release-0.7.0 or later version on yarn?

cdmikechen commented 2 years ago

@tonywang-sh You can not use hadoop-3.2.2, because there is no hadoop-3.2.2 profile in pom.xml. If you want to use hadoop 3.2.2, I think you need to change hadoop.version to 3.2.2. From 0.7.0, we only support k8s, so that you may use 0.6.0 for yarn as latest version.

tonywang-sh commented 2 years ago

Thanks for your reply. because submarine v0.6.0 on yarn needs TonY framework, how to set application conf for TonY to meet kerberos verification of hadoop? I didn't find TonY conf setting from the website, https://submarine.apache.org/docs/0.6.0/userDocs/yarn/YARNRuntimeGuide/.

cdmikechen commented 2 years ago

@tonywang-sh I have not tried kerberos, but in my experience, if you have logged into kerberos, hadoop will recognize kerberos authentication information by default. If you encounter problems when trying kerberos you can leave a comment here and I or someone else may be able to help you out.