集成KERBEROS报错

[li2008jun]

JobHistoryServer在xlearning-site.xml里面添加了

xlearning.history.keytab /var/run/cloudera-scm-agent/process/3001-hive-HIVESERVER2/hive.keytab

<property>
    <name>xlearning.history.principal</name>
    <value>hive/bd129118@MYCDH</value>
</property>

服务启动成功，但是运行demo的时候报错如下，集群各个机器上票据都正常 18/01/18 10:33:26 INFO Client: Application report for application_1516178233465_0044 (state: RUNNING) 18/01/18 10:33:26 WARN UserGroupInformation: PriviledgedActionException as:hive/bd129118@MYCDH (auth:KERBEROS) cause:org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[KERBEROS] 18/01/18 10:33:26 WARN Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[KERBEROS] 18/01/18 10:33:26 WARN UserGroupInformation: PriviledgedActionException as:hive/bd129118@MYCDH (auth:KERBEROS) cause:java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[KERBEROS] 18/01/18 10:33:26 WARN Client: Connecting to ResourceManager failed, try again later java.lang.reflect.UndeclaredThrowableException at com.sun.proxy.$Proxy21.fetchApplicationMessages(Unknown Source) at net.qihoo.xlearning.client.Client.waitCompleted(Client.java:682) at net.qihoo.xlearning.client.Client.submitAndMonitor(Client.java:643) at net.qihoo.xlearning.client.Client.main(Client.java:711) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: java.io.IOException: Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[KERBEROS]; Host Details : local host is: "bd129118/192.168.129.118"; destination host is: "bd129120":10079; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772) at org.apache.hadoop.ipc.Client.call(Client.java:1476) at org.apache.hadoop.ipc.Client.call(Client.java:1409) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:243) ... 10 more Caused by: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[KERBEROS] at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:688) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709) at org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:651) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:739) at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:376) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1525) at org.apache.hadoop.ipc.Client.call(Client.java:1448) ... 12 more Caused by: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[KERBEROS] at org.apache.hadoop.security.SaslRpcClient.selectSaslClient(SaslRpcClient.java:172) at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:396) at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:561) at org.apache.hadoop.ipc.Client$Connection.access$1900(Client.java:376) at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:731) at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:727) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:726) ... 15 more 18/01/18 10:33:27 INFO Client: Application report for application_1516178233465_0044 (state: RUNNING)

[jiarunying]

目前不支持kerberos验证

[li2008jun]

@jiarunying 可以给我一个常用邮箱，方便交流吗？

[jiarunying]

以下是xlearning的交流方式~~ Mail： g-xlearning-dev@360.cn QQ群：588356340

[lshmouse]

Same problem when submitting the tf example to a security hadoop cluster. Is there any plan to support the security cluster?

[iamabug]

请问现在XLearning支持Kerberos认证吗？

[zhaiyuyong]

支持kerberos很简单的，只要ContainerLaunchContext amContainer = ContainerLaunchContext.newInstance( localResources, appMasterEnv, appMasterLaunchcommands, null, null, null); 第二个null参数带上token就行

[LiuGuH]

可以参照提交的pr：https://github.com/Qihoo360/XLearning/pull/61/commits

[LiuGuH]

另外historyserver需要在配置文件中配置如下参数：xlearning.history.keytab ,xlearning.history.principal