Open pvbouwel opened 2 weeks ago
Every Spark application now stores it event logs which allows investigation after the fact.
While this data has a lot of value for debugging issues or deep dives it won´t be used in the majority of the cases.
It would be preferred to have a default of "spark.eventLog.enabled": "false" and to allow enablement (e.g. via job options)
"spark.eventLog.enabled": "false"
Every Spark application now stores it event logs which allows investigation after the fact.
While this data has a lot of value for debugging issues or deep dives it won´t be used in the majority of the cases.
It would be preferred to have a default of
"spark.eventLog.enabled": "false"and to allow enablement (e.g. via job options)