search

voltrondata / spark-substrait-gateway

Implements a gateway that speaks the SparkConnect protocol and drives a backend using Substrait (over ADBC Flight SQL).
Apache License 2.0
15 stars 8 forks source link

test: bump datafusion version and update test markers #80

Closed richtia closed 2 weeks ago

EpsilonPrime commented 2 weeks ago

Currently the tests are failing. The schema mismatch problem might be addressable with the following datafusion_backend change (I had to add unsigned int support to the server for the case where I saw this):

return df_result.to_arrow_table()

to:

        record_batch = df_result.collect()
        return pa.Table.from_batches(record_batch)