apache / polaris

The interoperable, open source catalog for Apache Iceberg
http://polaris.io/
Apache License 2.0
1.02k stars 101 forks source link

[BUG]Failed to call procedures in Spark #234

Closed flyrain closed 2 weeks ago

flyrain commented 2 weeks ago

Is this a possible security vulnerability?

Describe the bug

Failed to call procedures in Spark as the following message shows. It is due to a scala version mismatch between Spark and Iceberg lib.

scala> spark.sql("""call polaris.system.remove_orphan_files(table => 'db1.t1', dry_run => true)""").show();
org.apache.spark.sql.catalyst.parser.ParseException:
[PARSE_SYNTAX_ERROR] Syntax error at or near 'call'.(line 1, pos 0)

== SQL ==
call polaris.system.remove_orphan_files(table => 'db1.t1', dry_run => true)
^^^

  at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(parsers.scala:257)
  at org.apache.spark.sql.catalyst.parser.AbstractParser.parse(parsers.scala:98)
  at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:54)
  at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(AbstractSqlParser.scala:68)
  at org.apache.spark.sql.SparkSession.$anonfun$sql$5(SparkSession.scala:684)
  at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:138)
  at org.apache.spark.sql.SparkSession.$anonfun$sql$4(SparkSession.scala:683)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
  at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:682)
  at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:713)
  at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:744)
ebyhr commented 2 weeks ago

https://github.com/apache/polaris/pull/235 is merged. Can we close this issue now?