Open eitsupi opened 8 months ago
Yes, can you port that work to the polars-arrow
? It can iterator over a Vec<arrow::Chunk>
in that case.
Yes, can you port that work to the
polars-arrow
? It can iterator over aVec<arrow::Chunk>
in that case.
Sorry, but I may not have enough knowledge and time.
I think polars-arrow
does not have DataFrame
, but if we define an iterator in polars-arrow, does that mean we can also realize iterator for DataFrame
?
I think this is fixed by #17696?
I think this is fixed by #17696?
No, that is not C Stream interface, C interface.
Description
r-polars has the ability to export DataFrames via the Arrow C stream interface (pola-rs/r-polars#5, by @paleolimbot). https://arrow.apache.org/docs/format/CStreamInterface.html
Since this is not related to R, I believe that porting it through the polars crate will make it easier for all downstream projects to exchange DataFrames with other Arrow implementations. Conversely, it would be useful to be able to create DataFrames via the Arrow C stream interface.
For example, arrow-rs seems to exchange stream data with pyarrow (i.e., C++ libarrow) as follows: https://github.com/apache/arrow-rs/blob/121666e464170d7dce41bfd61de001a19affde72/arrow/src/pyarrow.rs#L387-L448