Closed XXwhite closed 3 weeks ago
社区无此环境, 你可以根据官网本地调试文档 进行本地调试寻找问题点及解决方案, 同时欢迎贡献您的 pr
经过查看相关代码: 在运行过程中将认证信息存在了 jvm 中(详见 org.dinky.gateway.yarn.YarnGateway#init
), 一旦进程释放, 所有都会销毁,可能造成以上问题
你可以对其进行优化并进行测试. 提供一个思路:
clusterConfigurationService
中testGateway
方法对每个集群配置重新走一遍认证的流程, 此思路仅供参考,更多细节可能需要考虑,如有其他方案请在此回复即可 @XXwhite
Search before asking
What happened
Caused by: org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1894) at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1805) at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1802) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1817) at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1812) at org.apache.hudi.util.StreamerUtil.tableExists(StreamerUtil.java:306) ... 175 more Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS] at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1629) at org.apache.hadoop.ipc.Client.call(Client.java:1575) at org.apache.hadoop.ipc.Client.call(Client.java:1472) at org.apache.hadoop.ipc.ProtobufRpcEngine2$Invoker.invoke(ProtobufRpcEngine2.java:245) at org.apache.hadoop.ipc.ProtobufRpcEngine2$Invoker.invoke(ProtobufRpcEngine2.java:131) at com.sun.proxy.$Proxy480.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:1008) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:435) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:170) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:162) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:100) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:366) at com.sun.proxy.$Proxy481.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1892) ... 181 more
What you expected to happen
After restarting the Dinky service, kerberos authentication becomes invalid,I found that resubmitting the task after saving the cluster configuration again in the registration center can solve this problem。 重启Dinky服务后,kerberos身份验证无效,我发现在注册中心再次保存集群配置后重新提交任务可以解决这个问题,这应该是个bug,期望修复~
How to reproduce
在kerberos认证的环境下,重启服务后提交任务就后认证失败的报错。
Anything else
No response
Version
1.1.0
Are you willing to submit PR?
Code of Conduct