Open Pooj2 opened 4 years ago
While reading the for example the Printing Application timeline, looking at 535 jobs it is hard to relate a number to an action in the Spark program. In the Spark UI there is a solution for this: set the spark.job.description and/or spark.jobGroup.id in the driver for every (eager) action you want the Spark executors to do (see the answers to https://stackoverflow.com/questions/39123314/how-to-add-custom-description-to-spark-job-for-displaying-in-spark-web-ui)
Is it possible to add an option to Sparklens to choose to report the spark.job.description and/or spark.jobGroup.id with its Job id?
While reading the for example the Printing Application timeline, looking at 535 jobs it is hard to relate a number to an action in the Spark program. In the Spark UI there is a solution for this: set the spark.job.description and/or spark.jobGroup.id in the driver for every (eager) action you want the Spark executors to do (see the answers to https://stackoverflow.com/questions/39123314/how-to-add-custom-description-to-spark-job-for-displaying-in-spark-web-ui)
Is it possible to add an option to Sparklens to choose to report the spark.job.description and/or spark.jobGroup.id with its Job id?