search

JuliaData / Feather.jl

Read and write feather files in pure Julia
https://juliadata.github.io/Feather.jl/stable
Other
109 stars 27 forks source link

Compression Support - Feather V2 Support #137

Closed ldsands closed 4 years ago

ldsands commented 4 years ago

I'm not sure if you've seen this yet but Arrow V17 is supporting feather V2 which includes compression support.

I'm putting this here more to put it on your radar I realize support for this will almost certainly take time. I use parquet for all of my data right now but given that all of my data is locally stored, feather V2 would probably be a better fit for what I need it for.

Thanks for all of your work!

ExpandingMan commented 4 years ago

Yup, feather V2 is really just the arrow IPC file format. I'm deep into the process of rewriting Arrow.jl and I can already read and mostly write a sizeable subset of the standard.

Since the new feather is just the arrow IPC format and doesn't require any additional code on top of arrow, we will like deprecate this package as legacy for reading Feather V1.

ldsands commented 4 years ago

Sounds great to me! I'll have to start following Arrow.jl. Thanks!