search

JahstreetOrg / spark-on-kubernetes-helm

Spark on Kubernetes infrastructure Helm charts repo
Apache License 2.0
199 stars 76 forks source link

Support Spark 3.0.0 #33

Closed duongnt closed 3 years ago

duongnt commented 4 years ago

@jahstreet Just to put this under your radar as Spark 3.0.0 has just been released, with I guess better support for K8s.

https://spark.apache.org/news/spark-3-0-0-released.html

jahstreet commented 4 years ago

Hi @duongnt , thanks you for putting it on the table. I'm tracking it as well and excited to try it out. 1 more blocker that we have for this repo charts is the release of Livy Spark 3 support https://github.com/apache/incubator-livy/pull/289 (scala 2.12 in particular). Once we have it - we can go. In the meantime I have plans on playing with it to see if it works.

duongnt commented 4 years ago

@jahstreet Looks like https://github.com/apache/incubator-livy/pull/289 has just landed.

jahstreet commented 4 years ago

@duongnt , on it now.

jahstreet commented 4 years ago

Please check the progress in #39 .