Support read-write parquet conversion to read-write arrow

oap-project / gazelle_plugin

Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.

Apache License 2.0

256 stars 77 forks source link

Support read-write parquet conversion to read-write arrow #1161

Open jackylee-ch opened 1 year ago

jackylee-ch commented 1 year ago

Is your feature request related to a problem or challenge? Please describe what you are trying to do. In our scenario, users usually read or write hive parquet tables, which different from arrow datasource tables, thus it's better to have a rule to convert ParquetFileFormat to ArrowFileFormat.