jorgecarleitao / arrow2

Transmute-free Rust library to work with the Arrow format
Apache License 2.0
1.07k stars 220 forks source link

feat: handle unaligned pointers in FFI #1535

Closed ritchie46 closed 11 months ago

ritchie46 commented 11 months ago

Arrow IPC can produce unaligned pointers and this will not be changed upstream.

This ensures we handle them gracefully. If the pointer is aligned, we use it zero copy. Otherwise we memcopy to a new vec.

See issue upstream: https://github.com/apache/arrow/issues/32276

And report of failure on polars: https://github.com/pola-rs/polars/issues/6315

codecov[bot] commented 11 months ago

Codecov Report

Patch coverage: 71.42% and project coverage change: +0.01% :tada:

Comparison is base (2ecd3e8) 83.44% compared to head (6e0fee3) 83.46%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1535 +/- ## ========================================== + Coverage 83.44% 83.46% +0.01% ========================================== Files 388 388 Lines 42141 42138 -3 ========================================== + Hits 35166 35169 +3 + Misses 6975 6969 -6 ``` | [Files Changed](https://app.codecov.io/gh/jorgecarleitao/arrow2/pull/1535?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao) | Coverage Δ | | |---|---|---| | [src/ffi/array.rs](https://app.codecov.io/gh/jorgecarleitao/arrow2/pull/1535?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao#diff-c3JjL2ZmaS9hcnJheS5ycw==) | `85.02% <71.42%> (+0.75%)` | :arrow_up: | ... and [5 files with indirect coverage changes](https://app.codecov.io/gh/jorgecarleitao/arrow2/pull/1535/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Jorge+Leitao)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.