search

voltrondata / spark-substrait-gateway

Implements a gateway that speaks the SparkConnect protocol and drives a backend using Substrait (over ADBC Flight SQL).
Apache License 2.0
15 stars 8 forks source link

feat: fix handling of datafusion local file renaming #77

Closed EpsilonPrime closed 3 weeks ago

EpsilonPrime commented 3 weeks ago

The new TPC-H dataset consists of multiple, unrelated files in the same directory. This updates the local file to virtual table rewrite code to handle that possibility.