[C++/Python] hdfs delegation token functions

apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

https://arrow.apache.org/

Apache License 2.0

14.62k stars 3.55k forks source link

[C++/Python] hdfs delegation token functions #17357

Open asfimport opened 7 years ago

asfimport commented 7 years ago

HDFS can create delegation tokens for an authenticated user, so that access to the file-system from other processes/machines can authenticate as that same user without having to use third-party identity systems (kerberos, etc.).

arrow-hdfs should provide the ability to accept, create, renew and cancel delegation tokens.

Reporter: Martin Durant / @martindurant

_{Note: This issue was originally created as ARROW-1321. Please see the migration documentation for further details.}

asfimport commented 3 years ago

Antoine Pitrou / @pitrou: It's not obvious how that would fit into the current FileSystem API. Feel free to explain your needs a bit more precisely.

FANNG1 commented 2 months ago

We had a requirement for supporting delegation tokens. Apache Gravitino is a centralized metadata manager, If an authorized user requests metadata, we want to deliver a delegation token from Gravitino, so the user can access the underlying HDFS without having to configure Kerberos.