search

eto-ai / rikai

Parquet-based ML data format optimized for working with unstructured data
https://rikai.readthedocs.io/en/latest/
Apache License 2.0
138 stars 19 forks source link

NoSuchMethodError for DBR 10.4 LTS #638

Closed da-liii closed 2 years ago

da-liii commented 2 years ago

Recently, Databricks upgraded the spark-catalysts jar in DBR 10.4 LTS.

(It is a Databricks bug or feature. Databricks Runtime is a blackbox.)

Affected Rikai Version: <= v0.1.8

22/04/20 08:31:50 ERROR DriverDaemon$: XXX Fatal uncaught exception. Terminating driver.
java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.parser.ParserUtils$.withOrigin(Lorg/antlr/v4/runtime/ParserRuleContext;Lscala/Function0;)Ljava/lang/Object;
    at ai.eto.rikai.sql.spark.parser.RikaiExtAstBuilder.visitSingleStatement(RikaiExtAstBuilder.scala:41)
    at ai.eto.rikai.sql.spark.parser.RikaiExtAstBuilder.visitSingleStatement(RikaiExtAstBuilder.scala:36)
    at ai.eto.rikai.sql.spark.parser.RikaiExtSqlBaseParser$SingleStatementContext.accept(RikaiExtSqlBaseParser.java:134)