spark jobs should handle end of execution

cnoam / technion-iem-ds_lab

anything relevant for the DS lab that can be made public

MIT License

1 stars 4 forks source link

Open cnoam opened 2 years ago

cnoam commented 2 years ago

a job sent to spark is completed hours after submission. I need a hook to know when it is done to manage book-keeping of concrruent jobs per user

cnoam commented 2 years ago

implemented a once per minute background thread that gets the applist.

idea for enhancment: send email to submitter when the application is no longer running for any reason