apache / arrow-nanoarrow

Helpers for Arrow C Data & Arrow C Stream interfaces
https://arrow.apache.org/nanoarrow
Apache License 2.0
169 stars 35 forks source link

refactor(r): Refactor ArrowArray(Stream) -> R Vector conversion #392

Open paleolimbot opened 7 months ago

paleolimbot commented 7 months ago

The initial version of converting ArrowArray (or stream of them) to R is implemented in C and is difficult to understand. Not only is this difficult because of the verbose C, the dispatch portion is implemented almost completely twice (once for a single array, once for an array stream). It is at a point currently where it is difficult for me, let alone an external contributor, to add features or fix bugs. Time to refactor!

This approach uses C++ classes/virtual method dispatch to handle the different types of vector conversions. This is similar to how the arrow R package does this except the Arrow R package uses the Arrow C++ converter infrastructure/heavy templating to do dispatch. Here we use a switch() and eat the per-batch and per-column virtual method call.

Work in progress!

codecov-commenter commented 7 months ago

Codecov Report

Attention: Patch coverage is 35.44474% with 479 lines in your changes are missing coverage. Please review.

Project coverage is 86.09%. Comparing base (c7a1236) to head (2afc7b8).

Files Patch % Lines
r/src/vctr_builder_int.h 2.98% 65 Missing :warning:
r/src/vctr_builder_int64.h 0.00% 59 Missing :warning:
r/src/vctr_builder_dbl.h 10.52% 51 Missing :warning:
r/src/vctr_builder.cc 73.21% 45 Missing :warning:
r/src/vctr_builder_difftime.h 37.50% 40 Missing :warning:
r/src/vctr_builder_other.h 11.11% 40 Missing :warning:
r/src/vctr_builder_base.h 30.35% 39 Missing :warning:
r/src/vctr_builder_lgl.h 5.26% 36 Missing :warning:
r/src/vctr_builder_chr.h 5.88% 32 Missing :warning:
r/src/vctr_builder_blob.h 4.00% 24 Missing :warning:
... and 5 more
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #392 +/- ## ========================================== - Coverage 88.74% 86.09% -2.66% ========================================== Files 81 97 +16 Lines 14398 15091 +693 ========================================== + Hits 12778 12993 +215 - Misses 1620 2098 +478 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.