palantir / k8s-spark-scheduler

A Kubernetes Scheduler Extender to provide gang scheduling support for Spark on Kubernetes
Apache License 2.0
175 stars 43 forks source link

add more explaination about memory annotations #173

Open chia7712 opened 3 years ago

chia7712 commented 3 years ago

related to https://github.com/palantir/k8s-spark-scheduler/issues/152#issuecomment-877627500

As I did not include the overhead when setting memory annotations, the scheduler tries to assign pods to nodes which have no enough (memory) resources. Also, I can observe following message from logs.

not enough capacity to reschedule the executor