search

kwai / blaze

Blazing-fast query execution engine speaks Apache Spark language and has Arrow-DataFusion at its core.
Apache License 2.0
1.3k stars 121 forks source link

Support Spark 3.1.x #553

Closed harveyyue closed 2 months ago

harveyyue commented 3 months ago

Since we use the 3.1.2 version almost three years and have no plan to upgrate it, so I expect the blaze to support Spark 3.1.x

harveyyue commented 3 months ago

@richox I can implement this feature for Spark 3.1.3

richox commented 2 months ago

you can just add version specs in EnableIf annotations in spark-extension-shims-spark3 module. i believe it is easy because we have already supported spark3.0/3.2