search

kubeflow / spark-operator

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
Apache License 2.0
2.79k stars 1.37k forks source link

custom serviceAccount for executor #885

Open cwei-bgl opened 4 years ago

cwei-bgl commented 4 years ago

Hi thanks for the work, spark-on-k8s-operator really makes thing easier.

It seems the operator currently does not support specifying a custom serviceAccount for executors. I tried both serviceAccount and serviceAccountName fields on executor node, which did not work and it was always being set to 'default'. I need a custom serviceAccount for executors so that they can access AWS resources. Alternatively, at least executors use the same serviceAccount from driver so that security can be controlled to a reasonable level. Thanks.

liyinan926 commented 4 years ago

The operator doesn't support custom service account for the executor pods. Spark on Kubernetes will support it in the future in 3.1 (see https://github.com/apache/spark/pull/27034). We can definitely add support for this.

cwei-bgl commented 4 years ago

Great! thanks @liyinan926

batCoder95 commented 4 years ago

The operator doesn't support custom service account for the executor pods. Spark on Kubernetes will support it in the future in 3.1 (see apache/spark#27034). We can definitely add support for this.

@liyinan926 - The pull request (27034) in above reply is in state closed. Does that mean that a new version (3.1) has been released with custom service account support ? Checking this in relation to the issue: IAM Role for Driver and Executor Pods? #1002

bbenzikry commented 3 years ago

Issue can be closed :) @batCoder95 This is now available via #1322

github-actions[bot] commented 1 week ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.