Failed to call procedures in Spark as the following message shows. It is due to a scala version mismatch between Spark and Iceberg lib.
scala> spark.sql("""call polaris.system.remove_orphan_files(table => 'db1.t1', dry_run => true)""").show();
org.apache.spark.sql.catalyst.parser.ParseException:
[PARSE_SYNTAX_ERROR] Syntax error at or near 'call'.(line 1, pos 0)
== SQL ==
call polaris.system.remove_orphan_files(table => 'db1.t1', dry_run => true)
^^^
at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(parsers.scala:257)
at org.apache.spark.sql.catalyst.parser.AbstractParser.parse(parsers.scala:98)
at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:54)
at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(AbstractSqlParser.scala:68)
at org.apache.spark.sql.SparkSession.$anonfun$sql$5(SparkSession.scala:684)
at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:138)
at org.apache.spark.sql.SparkSession.$anonfun$sql$4(SparkSession.scala:683)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:682)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:713)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:744)
Is this a possible security vulnerability?
Describe the bug
Failed to call procedures in Spark as the following message shows. It is due to a scala version mismatch between Spark and Iceberg lib.