search

jorgecarleitao / arrow2

Transmute-free Rust library to work with the Arrow format
Apache License 2.0
1.06k stars 222 forks source link

Not yet implemented: Decoding Int32 "PlainDictionary"-encoded optional parquet pages #1526

Open mdrach opened 1 year ago

mdrach commented 1 year ago

Hey, I encountered an issue similar to this one. Hit during 

            let arrays =
                column_iter_to_arrays(pages, types, field.clone(), None, row_group.num_rows())?;

pages is one of the pages in this file.

Error:

Not yet implemented: Decoding Int32 "PlainDictionary"-encoded optional  parquet pages.

Hope the fix is straightforward like the fix for the other issue. Please let me know if I can provide any additional information.

Using arrow2 0.17.0.

mdrach commented 1 year ago

Hitting these related issues on other parquet files as well.

  Not yet implemented: Decoding Double "PlainDictionary"-encoded required  parquet pages',
   Not yet implemented: Decoding FixedLenByteArray(16) "PlainDictionary"-encoded optional  parquet pages'

Link to file that produced second error.

avfeinberg commented 1 year ago

Experimented with the latest HEAD, still seeing this.