kylebarron / parquet-wasm

Rust-based WebAssembly bindings to read and write Apache Parquet data
https://kylebarron.dev/parquet-wasm/
Apache License 2.0
481 stars 19 forks source link

Update async API #510

Closed kylebarron closed 2 months ago

kylebarron commented 2 months ago

Change list

I couldn't get the row group selection to work in stream() here https://github.com/kylebarron/parquet-wasm/pull/510/files#diff-e1a77beecd2634c6c0489c20cc3cae036ed6668d62c4d47f00760ab60b0d404eR188-R192

I was hitting lifetime errors with having a Vec<usize> there that wouldn't live long enough for the stream.

@H-Plus-Time I'd like to get a release out in the next day or two, because otherwise I'll forget about it again and it'll never get released. I already did a lot of other cleanup, so I think it's just this and a little more README updates and then I'm ready to publish 0.6. I don't want to spend a lot more time on this. But I wanted to give you a heads up in case you wanted to make any more edits before the release!

github-actions[bot] commented 2 months ago

Asset Sizes

AssetUncompressed SizeCompressed Size
async_full/parquet_wasm_bg.wasm5.44MB $\color{green}\textbf{-18.6KB -0\%}$1.27MB $\color{green}\textbf{-5.32KB -0\%}$
slim/parquet_wasm_bg.wasm3.46MB $\color{red}\textbf{+1.57KB +0\%}$548KB $\color{red}\textbf{+565B +0\%}$
sync/parquet_wasm_bg.wasm4.74MB $\color{red}\textbf{+1.57KB +0\%}$1.04MB $\color{green}\textbf{-79B -0\%}$
H-Plus-Time commented 2 months ago

Yep, I'll give the read_stream row groups bit a crack. I've seen some quirks relating to the package exports too, will check a few import scenarios too.

kylebarron commented 2 months ago

I've seen some quirks relating to the package exports too, will check a few import scenarios too.

Would love bug reports if you have any!