[Bug] kyuubi using --proxy-user GSS init failed when hive.metastore.token.signature not empty

cs-lpc commented 1 year ago

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before asking

[X] I have searched in the issues and found no similar issues.

Describe the bug

According to the PR https://github.com/apache/spark/pull/42760, if the hive.metastore.token.signature parameter is not empty, we may also encounter authentication failures in Kyuubi.

Set the configuration bellow

<property>
   <name>hive.metastore.token.signature</name>
   <value>HIVE_DELEGATION_TOKEN</value>
</property>

Start kyuubi server with hive
Connet kyuubi with proxy user ，
Run sql insert into test11 values('a');

then the user will be hive to execute hdfs command

Affects Version(s)

master

Kyuubi Server Log Output

No response

Kyuubi Engine Log Output

No response

Kyuubi Server Configurations

<property>
       <name>hive.metastore.token.signature</name>
       <value>HIVE_DELEGATION_TOKEN</value>
    </property>

Kyuubi Engine Configurations

No response

Additional context

No response

Are you willing to submit PR?

[X] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
[ ] No. I cannot submit a PR at this time.

cs-lpc commented 1 year ago

Before Spark fixes this bug, Kyuubi should also be compatible with this behavior.

pan3793 commented 1 year ago

Do you have suggestions? Maybe forcibly overwriting hive.metastore.token.signature during spark-submit?

pan3793 commented 1 year ago

cc @zhouyifan279

cs-lpc commented 1 year ago

After merging https://github.com/apache/spark/pull/42760, directly overwriting this configuration may cause new exceptions. We need to be provied new configuration. By adjusting the default value of this configuration can adapt to different versions of Spark.

pan3793 commented 1 year ago

Yes, the overwriting behavior should be judged by the Spark version and also allowed to be disabled in case some vendor's Spark version backports it into lower Spark versions.

cxzl25 commented 1 year ago

It sounds like a misconfiguration, hive.metastore.token.signature should not be configured in hive-site.xml.

zzzzming95 commented 1 year ago

It sounds like a misconfiguration, hive.metastore.token.signature should not be configured in hive-site.xml.

hive.metastore.token.signature mean the namespace of the metastore token, the default value is empty. In hiveserver, they hardcode this conf's value.

https://github.com/apache/hive/blob/0fdf143cf775ccf909e092fda2eaef443e68fa23/service/src/java/org/apache/hive/service/auth/HiveAuthConstants.java#L44

But in some case we need to conf the configuration hive.metastore.token.signature in hive-site.xml.

cs-lpc commented 1 year ago

https://github.com/apache/kyuubi/pull/5261

apache / kyuubi