databricks / spark-sql-perf

Apache License 2.0
586 stars 407 forks source link

Spark 3.0.0 compile error #190

Open JamesPodogorski opened 4 years ago

JamesPodogorski commented 4 years ago

Build errors. spark 3.0.0

build.sbt snippet ... crossScalaVersions := Seq("2.11.12", "2.12.10") ... sparkVersion := "3.0.0"

[info] Compiling 66 Scala sources to /opt/spark-sql-perf/target/scala-2.12/classes... [info] 'compiler-interface' not yet compiled for Scala 2.12.10. Compiling... [info] Compilation completed in 18.491 s [error] /opt/spark-sql-perf/src/main/scala/com/databricks/spark/sql/perf/Benchmark.scala:338: value table is not a member of Seq[String] [error] case UnresolvedRelation(t) => t.table [error] ^ [error] /opt/spark-sql-perf/src/main/scala/com/databricks/spark/sql/perf/Query.scala:59: value table is not a member of Seq[String] [error] tableIdentifier.table [error] ^ [error] /opt/spark-sql-perf/src/main/scala/com/databricks/spark/sql/perf/Query.scala:94: missing argument list for method simpleString in class QueryPlan [error] Unapplied methods are only converted to functions when a function type is expected. [error] You can make this conversion explicit by writing simpleString _ or simpleString(_) instead of simpleString. [error] messages += s"Breakdown: ${node.simpleString}" [error] ^ [error] /opt/spark-sql-perf/src/main/scala/com/databricks/spark/sql/perf/Query.scala:107: missing argument list for method simpleString in class QueryPlan [error] Unapplied methods are only converted to functions when a function type is expected. [error] You can make this conversion explicit by writing simpleString _ or simpleString(_) instead of simpleString. [error] node.simpleString.replaceAll("#\d+", ""), [error] ^ [error] /opt/spark-sql-perf/src/main/scala/org/apache/spark/ml/ModelBuilderSSP.scala:62: not enough arguments for constructor NaiveBayesModel: (uid: String, pi: org.apache.spark.ml.linalg.Vector, theta: org.apache.spark.ml.linalg.Matrix, sigma: org.apache.spark.ml.linalg.Matrix)org.apache.spark.ml.classification.NaiveBayesModel. [error] Unspecified value parameter sigma. [error] val model = new NaiveBayesModel("naivebayes-uid", pi, theta) [error] ^ [error] /opt/spark-sql-perf/src/main/scala/org/apache/spark/ml/ModelBuilderSSP.scala:163: not enough arguments for method getCalculator: (impurity: String, stats: Array[Double], rawCount: Long)org.apache.spark.mllib.tree.impurity.ImpurityCalculator. [error] Unspecified value parameter rawCount. [error] ImpurityCalculator.getCalculator("variance", Array.fillDouble(0.0)) [error] ^ [error] /opt/spark-sql-perf/src/main/scala/org/apache/spark/ml/ModelBuilderSSP.scala:165: not enough arguments for method getCalculator: (impurity: String, stats: Array[Double], rawCount: Long)org.apache.spark.mllib.tree.impurity.ImpurityCalculator. [error] Unspecified value parameter rawCount. [error] ImpurityCalculator.getCalculator("gini", Array.fillDouble(0.0)) [error] ^ [error] 7 errors found [error] (compile:compileIncremental) Compilation failed [error] Total time: 178 s, completed Aug 12, 2020, 4:00:12 AM