Spark UI Glue 4.0 Logging Not Working?

[AlJohri]

I get this message and not more logs when I run the Glue 4.0 Spark UI container:

log4j:WARN No appenders could be found for logger (org.apache.spark.deploy.history.HistoryServer).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

[StevenTollanis]

@AlJohri I could solved adding the log4j attributes in the Dockerfile... and build it again

RUN echo $'\n\
spark.eventLog.enabled                      true\n\
spark.history.ui.port                       18080\n\
' > /opt/spark/conf/spark-defaults.conf

RUN echo $'\n\
 hadoop.root.logger=DEBUG, console\n\
log4j.rootLogger = DEBUG, console\n\
log4j.appender.console=org.apache.log4j.ConsoleAppender\n\
log4j.appender.console.target=System.out\n\
log4j.appender.console.layout=org.apache.log4j.PatternLayout\n\
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n\n\
' > /opt/spark/conf/log4j.properties

[moomindani]

Thank you for the feedback, and suggestion for the fix. I have added similar log4j.properties to fix the issue in this commit.