apache-spark-on-k8s / spark

Apache Spark enhanced with native Kubernetes scheduler back-end: NOTE this repository is being ARCHIVED as all new development for the kubernetes scheduler back-end is now on https://github.com/apache/spark/

https://spark.apache.org/

Apache License 2.0

612 stars 118 forks source link

Config for hard cpu limit on pods; default unlimited #356

Closed FANNG1 closed 7 years ago

FANNG1 commented 7 years ago

352

ash211 commented 7 years ago

Linking to https://github.com/apache-spark-on-k8s/spark/issues/38 which is the umbrella issue for fully-generic customization of things like this via user-provided yaml files instead of incrementally adding additional options for every k8s feature.

FANNG1 commented 7 years ago

patch updated, add spark.kubernetes.driver.limit.cores besides spark.kubernetes.executor.limit.cores

ash211 commented 7 years ago

rerun integration test please

ash211 commented 7 years ago

Hi @sandflee it looks like a recent commit caused a merge conflict in this PR. Can you please fix that and address the outstanding comments?

With those minor changes I think this PR is ready to merge

ash211 commented 7 years ago

Thank a bunch @sandflee for the contribution! Looking forward to jobs feeling faster for everyone