Support kerberos authentication for Hudi tables

[taqianw]

Hi,

My Hive and HDFS Cluster is Kerberos enabled, When I connect the hudi table through Presto's hive connector,I can show tables and describe hudi table's structure,but I can not select data ,there is error as following:

2020-09-30T10:27:29.063Z    DEBUG   query-execution-1   io.prestosql.execution.QueryStateMachine    Query 20200930_102728_00010_hnygc failed
io.prestosql.spi.PrestoException: Error checking Hoodie partition metadata for hdfs://xxx:8020/warehouse/tablespace/managed/hive/icosdata.db/avro_test2_mor
    at io.prestosql.plugin.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:223)
    at io.prestosql.plugin.hive.util.ResumableTasks$1.run(ResumableTasks.java:38)
    at io.prestosql.$gen.Presto_332____20200930_102455_2.run(Unknown Source)
    at io.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hudi.exception.HoodieException: Error checking Hoodie partition metadata for hdfs://xxx:8020/warehouse/tablespace/managed/hive/icosdata.db/avro_test2_mor
    at org.apache.hudi.common.model.HoodiePartitionMetadata.hasPartitionMetadata(HoodiePartitionMetadata.java:143)
    at org.apache.hudi.hadoop.HoodieParquetInputFormat.getTableMetaClient(HoodieParquetInputFormat.java:306)
    at org.apache.hudi.hadoop.InputPathHandler.parseInputPaths(InputPathHandler.java:98)
    at org.apache.hudi.hadoop.InputPathHandler.<init>(InputPathHandler.java:58)
    at org.apache.hudi.hadoop.HoodieParquetInputFormat.listStatus(HoodieParquetInputFormat.java:73)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:325)
    at io.prestosql.plugin.hive.BackgroundHiveSplitLoader.loadPartition(BackgroundHiveSplitLoader.java:407)
    at io.prestosql.plugin.hive.BackgroundHiveSplitLoader.loadSplits(BackgroundHiveSplitLoader.java:287)
    at io.prestosql.plugin.hive.BackgroundHiveSplitLoader.access$300(BackgroundHiveSplitLoader.java:107)
    at io.prestosql.plugin.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:216)
    ... 6 more
Caused by: java.io.IOException: DestHost:destPort xxx:8020 , LocalHost:localPort presto-coordinator/172.17.0.9:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]

But other hive tables have no problem.

Here are some of Presto's configurations:

• catalog/hive.properties:

  connector.name=hive-hadoop2
  hive.metastore.uri=thrift://xxx:9083
  hive.config.resources=/etc/hadoop/conf/core-site/core-site.xml,/etc/hadoop/conf/hdfs-site/hdfs-site.xml
  
  hive.metastore.authentication.type=KERBEROS
  hive.metastore.service.principal=hive/xxx@ICOS.CITY
  hive.metastore.client.principal=presto@ICOS.CITY 
  hive.metastore.client.keytab=/opt/presto.service.keytab
  
  hive.hdfs.authentication.type=KERBEROS
  hive.hdfs.impersonation.enabled=true
  hive.hdfs.presto.principal=presto@ICOS.CITY
  hive.hdfs.presto.keytab=/opt/presto.service.keytab

• config.properties:

  http-server.authentication.type=KERBEROS
  http.server.authentication.krb5.service-name=presto
  http-server.authentication.krb5.principal-hostname=ICOS.CITY
  http.server.authentication.krb5.keytab=/opt/presto.service.keytab
  http.authentication.krb5.config=/etc/krb5.conf
  
  http-server.https.enabled=true
  http-server.https.port=7778
  
  http-server.https.keystore.path=/opt/keystore.jks
  http-server.https.keystore.key=xxx

• presto version: prestosql-332/centos7/docker

Anyone has occured this problem?

Thanks.

[findepi]

@taqianw can you possibly test whether https://github.com/prestosql/presto/pull/5478 solves the issue for you?

[taqianw]

Thank you. It solved my problem,now presto works well.

[findepi]

@taqianw thanks for confirming

[liijiankang]

When the query time is long, such as more than an hour, this problem still occurs.