search

RumbleDB / rumble

⛈️ RumbleDB 1.21.0 "Hawthorn blossom" 🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more
http://rumbledb.org/
Other
211 stars 82 forks source link

Conflicting versions of ANTLR #68

Closed ingomueller-net closed 5 years ago

ingomueller-net commented 6 years ago

I am trying to run Sparksoniq on an HDInsight 3.6 cluster with Spark 2.2.0. I can start the shell, but running any query results in the following error:

ANTLR Tool version 4.6 used for code generation does not match the current runtime version 4.5.3ANTLR Runtime version 4.6 used for parser compilation does not match the current runtime version 4.5.3ANTLR Tool version 4.6 used for code generation does not match the current runtime version 4.5.3ANTLR Runtime version 4.6 used for parser compilation does not match the current runtime version 4.5.34

If I add some tracing options via spark-submit --conf 'spark.driver.extraJavaOptions=-verbose:class' --conf 'spark.executor.extraJavaOptions=-verbose:class ..., I get many lines similar to the following:

[Loaded org.antlr.v4.runtime.misc.ParseCancellationException from file:/usr/hdp/2.6.3.61-4/spark2/jars/antlr4-runtime-4.5.3.jar]

This resembles this issue of another Spark application. It seems like the Sparksoniq version I am using (the precompiled jsoniq-spark-app-0.9.1-jar-with-dependencies.jar from Github) expects ANTLR 4.6, but Spark loads 4.5.3. If Spark 2.3.0 uses ANTLR 4.7 like that other issue claims, changing the Spark version would fix the problem, but I haven't tried.

It could be that shading the dependencies could help.

ghislainfourny commented 6 years ago

This is simply because the jar was generated with ANTLR 4.6 and the Spark server has 4.5.3.

The solution, I think, is to offer several version of the release jars. We will think of that at the next release, in the meantime we will generate jars for you offline.

ghislainfourny commented 6 years ago

Note for the record: regenerating the jar using ANTLR 4.5.3 did the trick. We will provide both a jar for 4.5.3 (for Spark 2.0/2.1/2.2) and for 4.7 (for Spark 2.3) in the next release.

PraveenKumarPrecisely commented 1 year ago

I am still getting this issue. I will be grateful if there is any solution for the same.

ANTLR Tool version 4.11.1 used for code generation does not match the current runtime version 4.7.1ANTLR Runtime version 4.11.1 used for parser compilation does not match the current runtime version 4.7.123/04/11 14:57:30 ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 3)
java.lang.NullPointerException