search

apache / accumulo-proxy

Apache Accumulo Proxy
https://accumulo.apache.org
Apache License 2.0
9 stars 19 forks source link

ImportDirectory: java.lang.NullPointerException: invalid null input: name #82

Closed RangaSamudrala closed 1 month ago

RangaSamudrala commented 4 months ago

Hello I am trying to import data into Accumulo using this package. I have truncated the code the log to make it relevant. As one can see login is successful. I have been able to successfully invoke hdfs commands like hdfs dfs -rm -r -f hdfs://accumulo-hdfs-ha:8020/imports/custom_data_05222024 and hdfs dfs -put -f /imports/custom_data_05222024 hdfs://accumulo-hdfs-ha:8020/imports/custom_data_05222024. Even getting a list of Tablet servers works. However, invoking importDirectory fails with the error below.

proxy_connection = AccumuloProxyConnection(AccumuloProxyConnectionParams(hostname=proxy_host, port=proxy_port))
with proxy_connection:
     context = AccumuloProxyConnectionContext(proxy_connection)
     connector = context.create_connector('accumulo', secret)

     login = proxy_connection.client.login('accumulo', {'password': secret})
     if login:
          print(f"Accumulo login successful !!!")
     else:
          print(f'Accumulo login failed: {login}. Exiting...')
          sys.exit(1)

     client = proxy_connection.client

     tservers = client.getTabletServers(login)
     print(f'Tablet Servers: {tservers}')

     print(f'Accumulo import directory: {table_name} {hdfs_dir}  {failure_dir} {True}')
     client.importDirectory(login, table_name, hdfs_dir, failure_dir, True)

Log:

Accumulo login successful !!!

Tablet Servers: ['accumulo-tservers-2.accumulo-tservers.accumulo-dev.svc.cluster.local:9997', 'accumulo-tservers-4.accumulo-tservers.accumulo-dev.svc. cluster.local:9997', 'accumulo-tservers-0.accumulo-tservers.accumulo-dev.svc.cluster.local:9997', 'accumulo-tservers-3.accumulo-tservers.accumulo-dev. svc.cluster.local:9997', 'accumulo-tservers-1.accumulo-tservers.accumulo-dev.svc.cluster.local:9997']

Accumulo import directory: custom_data hdfs://accumulo-hdfs-ha:8020/imports/custom_data_05222024 None True

AccumuloException(msg='org.apache.hadoop.security.KerberosAuthException: failure to login: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name\n\t at jdk.security.auth/com.sun.security.auth.UnixPrincipal.(UnixPrincipal.java:67)\n\t at jdk.security.auth/com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:129)\n\t at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:747)\n\t at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:672)\n\t at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:670)\n\t at java.base/java.security.AccessController.doPrivileged(Native Method)\n\t at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:670)\n\t at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:581)\n\t at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2091)\n\t at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2001)\n\t at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:722)\n\t at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:672)\n\t at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:579)\n\t at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:3845)\n\t at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:3835)\n\t at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3623)\n\t at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:557)\n\t at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)\n\t at org.apache.accumulo.core.volume.VolumeImpl.(VolumeImpl.java:45)\n\t at org.apache.accumulo.core.volume.VolumeConfiguration.create(VolumeConfiguration.java:152)\n\t at org.apache.accumulo.core.volume.VolumeConfiguration.getVolume(VolumeConfiguration.java:39)\n\t at org.apache.accumulo.core.clientImpl.bulk.BulkImport.load(BulkImport.java:123)\n\t at org.apache.accumulo.proxy.ProxyServer.importDirectory(ProxyServer.java:1769)\n\t at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\t at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\t at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\t at java.base/java.lang.reflect.Method.invoke(Method.java:566)\n\t at org.apache.accumulo.core.trace.TraceUtil.lambda$wrapService$8(TraceUtil.java:232)\n\t at com.sun.proxy.$Proxy9.importDirectory(Unknown Source)\n\t at org.apache.accumulo.proxy.thrift.AccumuloProxy$Processor$importDirectory.getResult(AccumuloProxy.java:8073)\n\t at org.apache.accumulo.proxy.thrift.AccumuloProxy$Processor$importDirectory.getResult(AccumuloProxy.java:8052)\n\t at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)\n\t at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)\n\t at org.apache.accumulo.server.rpc.TimedProcessor.process(TimedProcessor.java:61)\n\t at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518)\n\t at org.apache.accumulo.server.rpc.CustomNonBlockingServer$CustomFrameBuffer.invoke(CustomNonBlockingServer.java:112)\n\t at org.apache.thrift.server.Invocation.run(Invocation.java:18)\n\t at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\t at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\t at org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)\n\t at java.base/java.lang.Thread.run(Thread.java:829)\n')

Please note that invoking importDirectory from within accumulo shell works fine.

ctubbsii commented 2 months ago

It would help to know what versions you're using (both Accumulo and the proxy). This proxy project is out of date and a version has not yet been released that works for Accumulo 2.x or later. Also, I don't think the current unreleased proxy code supports running the proxy with Kerberos, which I mention only because it seems that some of your error message involves Kerberos.

ctubbsii commented 1 month ago

Closing due to inactivity and lack of response.