Closed orthoxerox closed 3 months ago
Hi @orthoxerox, to confirm, both hive and delta catalogs are connecting to the same Hadoop Cluster?
Hi @imjalpreet, yes, it's the same Hadoop cluster.
I got the same Exception when use hudi connector in 0.286
connector.name=hudi hive.config.resources=/etc/hadoop/conf/hive-site.xml,/etc/hadoop/conf/hdfs-site.xml hive.metastore.uri=thrift://host:9083 hive.metastore.authentication.type=KERBEROS hive.metastore.service.principal=hive/_HOST@A.COM hive.metastore.client.principal=user1 hive.metastore.client.keytab=/etc/security/keytabs/user.keytab hive.hdfs.authentication.type=KERBEROS hive.hdfs.impersonation.enabled=true hive.hdfs.presto.principal=user1 hive.hdfs.presto.keytab=/etc/security/keytabs/user.keytab
@zhongtiancai Can you check once if your above configuration are working fine when you are connecting to hive connector?
And to confirm you have placed the required /etc/krb5.conf
on the coordinator host as well?
@agrawalreetika It can display Hive meta information, such as table information, but cannot access HDFS. and the /etc/krb5.conf is also added to the jvm config.
@zhongtiancai Could you please reach out to me over slack? needed some more setup information.
@agrawalreetika The program is deployed in the company and cannot connect to the Internet.
I tried change the connector from hudi to hive-hadoop2, and it can connect to HDFS normally. But hudi connector will first warn gss initialized fail, and then report 'Could not check if {path} is a valid table'.
I will try reading the source code and more documents to see if I have configured it incorrectly. Then provide feedback here
@agrawalreetika I solved the problem by delete the code on the krb5. conf file
#includedir /etc/krb5.conf.d/
. Because the default_ccached name is configured in this directory and KCM is used, this configuration may cause errors when uses Hadoop
Describe the problem you faced
Environment Description
Steps To Reproduce
Steps to reproduce the behavior:
Expected behavior
Presto returns the contents of the table
Additional context
The keytab we use looks like this:
Stacktrace