search

sal-openlab / datafusion-server

Rust DataFusion Server
MIT License
12 stars 2 forks source link

Avro support? #1

Closed rgutierrez2004 closed 6 months ago

rgutierrez2004 commented 7 months ago

DataFusion has built-in support for Avro, any plans in DataFusion Server to support Avro format?

neural-runner commented 7 months ago

I appreciate your feedback. I understand the importance of Avro and was planning to support it based on demand; however, I will now include it in our immediate plans. Nonetheless, Avro support in DataFusion is an option controlled by a feature flag, as it affects binary size. As DataFusion Server, we also want to enable it based on usage, likely through a feature flag, probably avro.

neural-runner commented 7 months ago

We have added support for the Avro data source format. In this release, we only support loading. Due to the inability to collect many Avro files, testing was primarily conducted with bin/data/popular-names.avro.

We hope you find it useful. Please let us know if you encounter any issues or have any other feedback. Thank you.