Apache Spark enhanced with native Kubernetes scheduler back-end: NOTE this repository is being ARCHIVED as all new development for the kubernetes scheduler back-end is now on https://github.com/apache/spark/
Each spark app creates a new hadoop-config configmap may waste lots of etcd storage and memory of k8s master components. Many of configmaps are duplicate assume lots of apps may share the same hdfs. I suggest adding an spark.kubernetes.configmap.hadoop.config config to make this possible.
Each spark app creates a new hadoop-config configmap may waste lots of etcd storage and memory of k8s master components. Many of configmaps are duplicate assume lots of apps may share the same hdfs. I suggest adding an spark.kubernetes.configmap.hadoop.config config to make this possible.