k8s: event based job tracking?

[jdries]

On kubernetes, there is an event api that can inform us about spark application transitions: kubectl --kubeconfig dev.yml -n spark-jobs get events --field-selector involvedObject.kind=SparkApplication --sort-by=.metadata.creationTimestamp

This might allow us to improve job tracking:

• more robust as we can see and log all changes
• more lightweight, only act upon actual changes
• reduced delay between event and update in openEO state

[soxofaan]

Maybe I'm misunderstanding, but you still have to run this get events command on regular basis, right? So you still need a loop or cron system to poll this event listing, which means it won't help much with

reduced delay between event and update in openEO state

(unless the polling is considerably lighter, so you can increase the poll frequency)