apache / kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
https://kyuubi.apache.org/
Apache License 2.0
2.11k stars 916 forks source link

[Improvement] Support kill yarn application through proxy user #2698

Open ulysses-you opened 2 years ago

ulysses-you commented 2 years ago

Code of Conduct

Search before asking

What would you like to be improved?

Kyuubi always uses the current user to do yarnClient.killApplication, e.g. hive. But if the user has no permission to do kill, the error will be thrown.

Caused by: java.security.AccessControlException: User hive cannot perform operation MODIFY_APP on application_1649986670278_3655140
    at org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69) ~[kyuubi-common_2.12-1.5.1-incubating.jar:1.5.1-incubating]
    at org.apache.kyuubi.engine.ProcBuilder.$anonfun$start$1(ProcBuilder.scala:165) ~[kyuubi-server_2.12-1.5.1-incubating.jar:1.5.1-incubating]

So, if the engine is created and started through proxy user, we can also use proxy user to do the kill action.

How should we improve?

No response

Are you willing to submit PR?

ulysses-you commented 2 years ago

@yaooqinn @wForget @turboFei what do you think about this issue ?

yaooqinn commented 2 years ago

yes, better to use app user to kill but i am also wondering whether it is common case that a user with create privilege will also have delete privilege on yarn

yaooqinn commented 2 years ago

what are livy or other platform’s behaviors?

ulysses-you commented 2 years ago

I searched some related code in Livy, it does not support kill yarn application through proxy user. I think it assumes the super user also has the permission of MODIFY_APP for yarn..

turboFei commented 2 years ago

I tested and did not meet this issue.

because for me, kyuubi is in hadoop group, it is a super user.

yaooqinn commented 2 years ago

a superuser seems much easier to deploy for internal app management.

turboFei commented 2 years ago

a superuser seems much easier to deploy for internal app management.

yes