search

apache-spark-on-k8s / spark

Apache Spark enhanced with native Kubernetes scheduler back-end: NOTE this repository is being ARCHIVED as all new development for the kubernetes scheduler back-end is now on https://github.com/apache/spark/
https://spark.apache.org/
Apache License 2.0
612 stars 118 forks source link

Allow configuration to set environment variables on driver and executor #424

Closed tangzhankun closed 7 years ago

tangzhankun commented 7 years ago

What changes were proposed in this pull request?

Two configurations added as below to support setting environment variables into driver and executors:

--conf spark.executorEnv.[EnvironmentVariableName] --conf spark.kubernetes.driverEnv.[EnvironmentVariableName]

How was this patch tested?

For driver, unit test case is added. For both driver and executor, manually integration test against the k8s cluster is performed.

liyinan926 commented 7 years ago

LGTM.

erikerlandson commented 7 years ago

@tangzhankun thanks for rebasing