search

mukunku / ParquetViewer

Simple Windows desktop application for viewing & querying Apache Parquet files
GNU General Public License v3.0
687 stars 82 forks source link

[BUG] Error when opening file containing columns of LIST type #73

Closed mughees-wyne closed 1 year ago

mughees-wyne commented 1 year ago

Parquet Viewer Version 2.5.1

Where was the parquet file created? pyarrow

Sample File test_file_20230120_common_ts.zip

Describe the bug The Parquet Viewer is not able to open a file that contains columns of LIST type. In the attached file "object_data__obj_angles" is a column of LIST type but as you see in the screenshot it complains that this column doesn't exist (don't know why). However, v2.4 mentioned at https://github.com/mukunku/ParquetViewer/issues/33#issuecomment-860229328 is able to load this file without issues.

Screenshots v2.5.1 image

v2.4 image

mukunku commented 1 year ago

I was able to add non-nested list support in v2.4. But after trying to handle nested lists I kind of gave up because implementing the dremel algorithm is beyond my skillset. Maybe I can look into adding non-nested list support again but it's known that list types aren't supported in the app as mentioned in the readme.

mughees-wyne commented 1 year ago

I was able to add non-nested list support in v2.4. But after trying to handle nested lists I kind of gave up because implementing the dremel algorithm is beyond my skillset. Maybe I can look into adding non-nested list support again but it's known that list types aren't supported in the app as mentioned in the readme.

Non-nested list support would be better than having nothing at all. It should be enough for many simple use cases.

mukunku commented 1 year ago

Non-nested list support is available now with v2.7.0

Going to close out this issue. Please feel free to re-open if you experience any issues.