search

big-data-europe / docker-spark

Apache Spark docker image
2.04k stars 698 forks source link

Should export SPARK_HOME before loading /spark/sbin/spark-config.sh #110

Closed ideal closed 3 years ago

ideal commented 3 years ago

File /spark/sbin/spark-config.sh needs SPARK_HOME, if not defined, it will define a bad one and spawn a java process like:

/usr/lib/jvm/java-1.8-openjdk/jre/bin/java -cp ///conf/:/spark/jars/* -Xmx1g ...

Here ///conf is a wrong path, it should be /spark/conf.