apache / arrow-nanoarrow

Helpers for Arrow C Data & Arrow C Stream interfaces
https://arrow.apache.org/nanoarrow
Apache License 2.0
169 stars 35 forks source link

[R] as_nanoarrow_array for nanoarrow_array_stream? #371

Closed eitsupi closed 8 months ago

eitsupi commented 8 months ago

I wonder collect_array_stream can be accessed via as_nanoarrow_array(<nanoarrow_array_stream>)

paleolimbot commented 8 months ago

as_nanoarrow_array() for a nanoarrow_array_stream would involve concatenating (i.e., a copy), and that isn't something nanoarrow knows how to do. You could, as you noted, go through an R vector, but that's slow and I'd prefer to have the "chunked" case handled. The chunked case is the most common one anyway (column in an Arrow Table, polars Series).

eitsupi commented 8 months ago

that isn't something nanoarrow knows how to do

I see, I understand. And sorry, I completely misunderstood what collect_array_stream does. It returns a LIST, not an array.......

Thanks for the quick reply!