search

dmlc / wormhole

Deprecated
Other
338 stars 153 forks source link

run dmlc yarn error, "failure to login" #47

Open robbine opened 8 years ago

robbine commented 8 years ago

hi all, i tried to run agaricus example on yarn. The following exception is thrown:

~/platform/java-1.8.0//bin/java -cp /home/hadoop/bin/hadoop classpath:tracker/../yarn//dmlc-yarn.jar org.apache.hadoop.yarn.dmlc.Client -file tracker/../yarn//dmlc-yarn.jar -file tracker/../yarn//run_hdfs_prog.py -file bin/xgboost.dmlc -jobname DMLC[nworker=4]:xgboost.dmlc -tempdir /tmp -queue default ./run_hdfs_prog.py ./xgboost.dmlc 15/11/25 20:01:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Exception in thread "main" java.io.IOException: failure to login at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:782) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:734) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:607) at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:2748) at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:2740) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2606) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:167) at org.apache.hadoop.yarn.dmlc.Client.(Client.java:73) at org.apache.hadoop.yarn.dmlc.Client.main(Client.java:322) Caused by: javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: Illegal principal name XXX@HADOOP.XXX.COM at org.apache.hadoop.security.User.(User.java:50) at org.apache.hadoop.security.User.(User.java:43) at org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:179) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) at javax.security.auth.login.LoginContext.login(LoginContext.java:588) at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:757) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:734) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:607) at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:2748) at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:2740) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2606) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:167) at org.apache.hadoop.yarn.dmlc.Client.(Client.java:73) at org.apache.hadoop.yarn.dmlc.Client.main(Client.java:322) Caused by: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to XXX@HADOOP.XXX.COM at org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:389) at org.apache.hadoop.security.User.(User.java:48) ... 23 more

at javax.security.auth.login.LoginContext.invoke(LoginContext.java:856)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:588)
at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:757)
... 9 more

Exception in thread Thread-2: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 552, in bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 505, in run self.__target(_self.args, *_self.__kwargs) File "tracker/dmlc_yarn.py", line 191, in run subprocess.check_call(cmd, shell = True, env = env) File "/usr/lib/python2.7/subprocess.py", line 511, in check_call raise CalledProcessError(retcode, cmd)

robbine commented 8 years ago

seems a Kerberos problem. The question is i've already used kinit.

robbine commented 8 years ago

Here is the code where exception happened: 73 dfs = FileSystem.get(conf); 74 userName = UserGroupInformation.getCurrentUser().getShortUserName(); 75 credentials = UserGroupInformation.getCurrentUser().getCredentials();