Update to DataFusion 33

vega / vegafusion

Serverside scaling for Vega and Altair visualizations

https://vegafusion.io

BSD 3-Clause "New" or "Revised" License

303 stars 15 forks source link

Update to DataFusion 33 #424

Closed jonmmease closed 7 months ago

jonmmease commented 7 months ago

WIP update to DataFusion 33.

Known Issues:

There seems to be a regression in arrow-rs when decoding string columns that contain numbers. See https://github.com/apache/arrow-rs/issues/5095. This breaks all of the tests that use the Movies dataset, because some of the movie titles are numbers.

jonmmease commented 7 months ago

The arrow issue was fixed and should be released in arrow-rs 50. We'll have to wait until DataFusion updates to arrow 50+, which may be a couple of months.