search

sunchao / parquet-rs

Apache Parquet implementation in Rust
Apache License 2.0
149 stars 20 forks source link

Limit writing row groups based either on size or number of records #165

Open sadikovi opened 5 years ago

sadikovi commented 5 years ago

This is derived from #116.

Currently we do not limit row groups size in any way, even though we have options in WriterProperties for it, like max row group size. Parquet-mr and parquet-cpp already handle that, we could borrow the mechanism from either implementation.