apache / arrow-rs

Official Rust implementation of Apache Arrow
https://arrow.apache.org/
Apache License 2.0
2.62k stars 802 forks source link

Speed up `filter_run_end_array` #6712

Closed Dandandan closed 6 days ago

Dandandan commented 1 week ago

Which issue does this PR close?

filter run array (kept 1/2)
                        time:   [265.17 µs 266.54 µs 268.32 µs]
                        change: [-48.180% -47.816% -47.483%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe

filter run array high selectivity (kept 1023/1024)
                        time:   [181.62 µs 182.38 µs 183.26 µs]
                        change: [-17.967% -17.356% -16.742%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe

filter run array low selectivity (kept 1/1024)
                        time:   [137.40 µs 137.83 µs 138.37 µs]
                        change: [-20.107% -19.685% -19.217%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) high mild
  3 (3.00%) high severe

Closes #.

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

Dandandan commented 1 week ago

FYI @delamarch3