Open janekdb opened 3 years ago
Just change Scala and Spark versions in build.sbt and rebuild the package with sbt compile && sbt package
Scala 2.12 and Spark 2.4.0 worked for me.
Does Qubole have any plans to release a build like that to spark-packages?
Hey, any update on this topic?
Scala 2.12 support would be nice. For myself, i'm using GCP DataProc with an image of Apache Spark 3.1.3 and Scala 2.12.14. https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-2.0
Having sparklens as a supported listener solution in this environment with an emailed report would be really nice, because my organization tends to restrict FW rules aggressively that makes it very difficult to utilize many of the UI based diagnostic tools.
Just change Scala and Spark versions in build.sbt and rebuild the package with sbt compile && sbt package Scala 2.12 and Spark 2.4.0 worked for me.
@oleksiilopasov thanks, but this did not work for me. Compilation failed, SparkHadoopUtil is not longer accessible, along with some other errors.
Anyone tried building sparklens with Spark 3.5.1? First I had to build and publish sbt-spark-package 0.2.4 locally but then I get this error:
[error] /Users/antonioye/code/sparklens/src/main/scala/com/qubole/sparklens/helper/HDFSConfigHelper.scala:28: object SparkHadoopUtil in package deploy cannot be accessed in package org.apache.spark.deploy
Hi,
Any plans to release for Scala 2.12?
Scala 2.13 and Spark 3 would be good to have support for.
Thanks, Janek