search

bluedenim / log4j-s3-search

Log4j appender with S3, Azure, Google Cloud, and search publishing
MIT License
65 stars 44 forks source link

Spark job Running forever after completed #155

Open sontt710 opened 1 week ago

sontt710 commented 1 week ago

Hello,

I try to use appender with the following version to submit spark job (spark-pi example) on k8s, but the job running forever and not shutdown after complete, can you help me to check

Version

Spark 3.3.4 Log4j2 Appender 5.3.2 hadoop-aws-3.3.2.jar aws-java-sdk-bundle-1.11.1026.jar aws-java-sdk-s3-1.12.772.jar

Driver log

image
bluedenim commented 1 week ago

If you are quitting a process, see this to shut down: https://github.com/bluedenim/log4j-s3-search-samples/blob/master/appender-log4j2-sample/src/main/java/com/van/example/Main.java#L43

sontt710 commented 1 week ago

Thanks @bluedenim, I understand, I will have to shut it down myself in the application, there's no way to configure it from the log4j2 XML file?

bluedenim commented 1 week ago

There is no easy way for the event log thread to know when your program should end. Only your program knows when it's time to end.