search

NVIDIA / spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs
https://nvidia.github.io/spark-rapids
Apache License 2.0
797 stars 232 forks source link

[BUG] Build on Databricks failed with GpuGetJsonObject.scala:19: object parsing is not a member of package util #10527

Closed NvTimLiu closed 7 months ago

NvTimLiu commented 7 months ago

Describe the bug Build failed on branch-24.04 Databricks build with GpuGetJsonObject.scala:19: object parsing is not a member of package util

I can not repro this build failure on local host out of Databricks runtime.

 [INFO] --- scala-maven-plugin:3.4.4:compile (scala-compile-first) @ rapids-4-spark-sql_2.12 ---
 [INFO] Using incremental compilation
 [WARNING] Pruning sources from previous analysis, due to incompatible CompileSetup.
 [INFO] Compiling 456 Scala sources and 58 Java sources to /home/ubuntu/spark-rapids/sql-plugin/target/spark341db/classes...
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:19: object parsing is not a member of package util
 [ERROR] import scala.util.parsing.combinator.RegexParsers
 [ERROR]                   ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:38: not found: type RegexParsers
 [ERROR] private[this] object JsonPathParser extends RegexParsers {
 [ERROR]                                             ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:41: not found: type Parser
 [ERROR]   def root: Parser[Char] = '$'
 [ERROR]             ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:43: not found: type Parser
 [ERROR]   def long: Parser[Long] = "\\d+".r ^? {
 [ERROR]             ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:43: value ^? is not a member of scala.util.matching.Regex
 [ERROR]   def long: Parser[Long] = "\\d+".r ^? {
 [ERROR]                                     ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:48: not found: type Parser
 [ERROR]   def subscript: Parser[List[PathInstruction]] =
 [ERROR]                  ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:50: value ~> is not a member of Char
 [ERROR]       operand <- '[' ~> ('*' ^^^ Wildcard | long ^^ Index) <~ ']'
 [ERROR]                      ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:50: value ^^^ is not a member of Char
 [ERROR]       operand <- '[' ~> ('*' ^^^ Wildcard | long ^^ Index) <~ ']'
 [ERROR]                              ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:56: not found: type Parser
 [ERROR]   def named: Parser[List[PathInstruction]] =
 [ERROR]              ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:58: value ~> is not a member of Char
 [ERROR]       name <- '.' ~> "[^\\.\\[]+".r | "['" ~> "[^\\'\\?]+".r <~ "']"
 [ERROR]                   ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:58: value ~> is not a member of String
 [ERROR]       name <- '.' ~> "[^\\.\\[]+".r | "['" ~> "[^\\'\\?]+".r <~ "']"
 [ERROR]                                            ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:64: not found: type Parser
 [ERROR]   def wildcard: Parser[List[PathInstruction]] =
 [ERROR]                 ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:65: value | is not a member of String
 [ERROR]     (".*" | "['*']") ^^^ List(Wildcard)
 [ERROR]           ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:67: not found: type Parser
 [ERROR]   def node: Parser[List[PathInstruction]] =
 [ERROR]             ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:72: not found: type Parser
 [ERROR]   val expression: Parser[List[PathInstruction]] = {
 [ERROR]                   ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:73: not found: value phrase
 [ERROR]     phrase(root ~> rep(node) ^^ (x => x.flatten))
 [ERROR]     ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:77: value parseAll is not a member of object com.nvidia.spark.rapids.JsonPathParser
 [ERROR]     this.parseAll(expression, str) match {
 [ERROR]          ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:78: not found: value Success
 [ERROR]       case Success(result, _) =>
 [ERROR]            ^
 [ERROR] /home/ubuntu/spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala:19: Unused import
 [ERROR] import scala.util.parsing.combinator.RegexParsers
 [ERROR]                                      ^
 [ERROR] 19 errors found
revans2 commented 7 months ago

We probably just have not pulled in the correct dependency jar from databricks. It is not a core part of scala apparently.

https://www.scala-lang.org/api/2.12.4/scala-parser-combinators/scala/util/parsing/combinator/Parsers.html