search

ytsaurus / ytsaurus-spyt

YTsaurus SPYT provides an integration with Apache Spark
Apache License 2.0
12 stars 4 forks source link

ACL support #14

Closed MrSandmanRUS closed 3 months ago

MrSandmanRUS commented 3 months ago

At the moment, when launching spyt-job, it is impossible to specify which other users can work with the Job in order to view logs, be able to kill the Job, and so on.

It will be great to be able to specify ACL settings when spark-submit.

Alexvsalexvsalex commented 3 months ago

What type of submit do you use (Standalone or YTsaurus scheduler)?

MrSandmanRUS commented 3 months ago

I am trying spark-submit --master ytsaurus://$YT_PROXY ...

Now I have not found an opportunity to transfer the ACL.

Alexvsalexvsalex commented 3 months ago

It exists. It's available in the last release 2.1.0. Please, look here: https://ytsaurus.tech/docs/en/user-guide/data-processing/spyt/thesaurus/gpu-usage#configuring-the-gpu-for-direct-execution. There is example of redefining pool using spark.ytsaurus.executor.operation.parameters. You can write kind of:

spark-submit ... --conf spark.ytsaurus.driver.operation.parameters="{acl=[...]}" --conf spark.ytsaurus.executor.operation.parameters="{acl=[...]}"
MrSandmanRUS commented 3 months ago

This is great news, apparently I missed it, thanks for the link. We will try to update our version.