For consideration, here's a PR that makes the feather R package import arrow, like how the Python implementation is just a wrapper around pyarrow. Aside from reducing maintenance burden, this will help us ensure that the Feather implementation in the arrow package meets all (tested) expectations of the feather package. Indeed, there were a few minor features missing from arrow version 0.17 that this exposed, and they were all addressed in the recent 1.0.0 release.
Most feather APIs are preserved in spirit, but key changes in this PR:
The feather class, which allowed for accessing data in a Feather file without reading it all into R, has been replaced by an arrow::Table backed by the memory-mapped Feather file. This should preserve the intent of the original implementation but with much richer functionality. One behavior change that results though is that slicing/extracting from the Table results in another arrow Table, so the data aren't pulled into a data.frame until you as.data.frame() them.
feather_metadata also now does the same. There didn't seem to be tests for feather_metadata() so I don't know if there are some other expected behaviors.
For consideration, here's a PR that makes the
feather
R package importarrow
, like how the Python implementation is just a wrapper aroundpyarrow
. Aside from reducing maintenance burden, this will help us ensure that the Feather implementation in thearrow
package meets all (tested) expectations of thefeather
package. Indeed, there were a few minor features missing fromarrow
version 0.17 that this exposed, and they were all addressed in the recent1.0.0
release.Most
feather
APIs are preserved in spirit, but key changes in this PR:feather
class, which allowed for accessing data in a Feather file without reading it all into R, has been replaced by anarrow::Table
backed by the memory-mapped Feather file. This should preserve the intent of the original implementation but with much richer functionality. One behavior change that results though is that slicing/extracting from the Table results in another arrow Table, so the data aren't pulled into a data.frame until youas.data.frame()
them.feather_metadata
also now does the same. There didn't seem to be tests forfeather_metadata()
so I don't know if there are some other expected behaviors.