The documentation for Spark History Server show that there is support for S3 for storing Spark job logs and the suggestion that HDFS support may be added in the future. We should add support for HDFS for log storage as an alternative to S3 object storage.
The documentation for Spark History Server show that there is support for S3 for storing Spark job logs and the suggestion that HDFS support may be added in the future. We should add support for HDFS for log storage as an alternative to S3 object storage.