Feature: Add support for Arrow's new run-end encoding

jorgecarleitao / arrow2

Transmute-free Rust library to work with the Arrow format

Apache License 2.0

1.06k stars 222 forks source link

Feature: Add support for Arrow's new run-end encoding #1413

Open talawahtech opened 1 year ago

talawahtech commented 1 year ago

The arrow format has a brand new run-end encoded layout. It is a variation of run-length encoding, as such it allows repeated values to be encoded in a space efficient manner that supports fast data processing.

Corresponding arrow-rs issue