I can currently write parquet files in a hive partition scheme using "write_parquet(file, partition_by=['a', 'b'])".
The partition_by parameter is currently not available for the "sink_parquet" function. This would be a huge benefit for use to not need to collect the whole LazyFrame before writing it.
aut0clave
commented
1 week ago
Description
I can currently write parquet files in a hive partition scheme using "write_parquet(file, partition_by=['a', 'b'])". The partition_by parameter is currently not available for the "sink_parquet" function. This would be a huge benefit for use to not need to collect the whole LazyFrame before writing it.