search

delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
7.63k stars 1.71k forks source link

[Spark] Fix delta-spark test log4j #3700

Closed scottsand-db closed 2 months ago

scottsand-db commented 2 months ago

Which Delta project/connector is this regarding?

Description

https://github.com/delta-io/delta/pull/3146 added support for spark master structured logging, but broke the test logging for delta-spark against spark 3.5. This PR fixes that.

The broken logging looked like this (https://github.com/delta-io/delta/actions/runs/10856009436/job/30129811815):

ERROR StatusConsoleListener Unable to locate plugin type for JsonTemplateLayout
ERROR StatusConsoleListener Unable to locate plugin for JsonTemplateLayout
ERROR StatusConsoleListener Could not create plugin of type class org.apache.logging.log4j.core.appender.FileAppender for element File: java.lang.NullPointerException
 java.lang.NullPointerException
    at org.apache.logging.log4j.core.config.plugins.visitors.PluginElementVisitor.findNamedNode(PluginElementVisitor.java:104)

How was this patch tested?

GitHub CI tests.

Does this PR introduce any user-facing changes?

No.