Update to Spark 3.1.2 and Hadoop 3.2.2 - Githubissues

gettyimages / docker-spark

Docker build for Apache Spark

MIT License

679 stars 369 forks source link

Update to Spark 3.1.2 and Hadoop 3.2.2 #74

Open kennyhe opened 2 years ago

kennyhe commented 2 years ago

Updated Dockerfile dependencies to allow running Spark 3.1.2:

Debian base image from stretch to buster.
Python 3.7
Java 11
Hadoop 3.2.2
Spark 3.1.2
Download Hadoop/Spark from CDN (faster)