search

apache-spark-on-k8s / spark

Apache Spark enhanced with native Kubernetes scheduler back-end: NOTE this repository is being ARCHIVED as all new development for the kubernetes scheduler back-end is now on https://github.com/apache/spark/
https://spark.apache.org/
Apache License 2.0
612 stars 118 forks source link

Can't print log of spark and scala-logging #569

Open livelxw opened 6 years ago

livelxw commented 6 years ago

I can't see the logs of spark and scala-logging when submit my application to the k8s. However, I can see the logs if I use println() function. I tried to put a log4j properties file in resource folder but it didn't work. What's wrong?

liyinan926 commented 6 years ago

I'm assuming you used kubectl logs <driver pod name> to get logs of the driver. This only works if the logs are written to stdout/stderr. See https://kubernetes.io/docs/concepts/cluster-administration/logging/#logging-at-the-node-level.