I added in support to map multiple CSV files into a single parquet file in order to increase row group compression and decrease spectrum query times.
While mapping a single CSV to a single parquet file is a fine default for unloading full tables, for unloading partitions it tended to produce too many small parquet files.
Are you open to PRs?
I added in support to map multiple CSV files into a single parquet file in order to increase row group compression and decrease spectrum query times.
While mapping a single CSV to a single parquet file is a fine default for unloading full tables, for unloading partitions it tended to produce too many small parquet files.