palantir / k8s-spark-scheduler

A Kubernetes Scheduler Extender to provide gang scheduling support for Spark on Kubernetes
Apache License 2.0
175 stars 43 forks source link

Log spark app id when scheduling pod #224

Closed Chrisbattarbee closed 1 year ago

Chrisbattarbee commented 1 year ago

We now log the spark app id when scheduling a pod, this makes it easier to find scheduling log lines for specific applications and makes debugging easier.

This label is always guaranteed to be set or else scheduling will fail later when we create resource reservations as the name of the resource reservation is equal to the spark-app-id label value.