Closed AlJohri closed 1 year ago
@AlJohri I could solved adding the log4j attributes in the Dockerfile... and build it again
RUN echo $'\n\
spark.eventLog.enabled true\n\
spark.history.ui.port 18080\n\
' > /opt/spark/conf/spark-defaults.conf
RUN echo $'\n\
hadoop.root.logger=DEBUG, console\n\
log4j.rootLogger = DEBUG, console\n\
log4j.appender.console=org.apache.log4j.ConsoleAppender\n\
log4j.appender.console.target=System.out\n\
log4j.appender.console.layout=org.apache.log4j.PatternLayout\n\
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n\n\
' > /opt/spark/conf/log4j.properties
Thank you for the feedback, and suggestion for the fix. I have added similar log4j.properties to fix the issue in this commit.
I get this message and not more logs when I run the Glue 4.0 Spark UI container: