search

voltrondata / spark-substrait-gateway

Implements a gateway that speaks the SparkConnect protocol and drives a backend using Substrait (over ADBC Flight SQL).
Apache License 2.0
15 stars 8 forks source link

Support inline SQL expressions #49

Open EpsilonPrime opened 1 month ago

EpsilonPrime commented 1 month ago

Spark supports using arbitrary SQL expressions within a plan using the expr function.

EpsilonPrime commented 1 month ago

Support for SQL is currently limited but there are plans to expand it by utilizing the recently updated Ibis-Substrait package.