mukunku / ParquetViewer

Simple Windows desktop application for viewing & querying Apache Parquet files
GNU General Public License v3.0
687 stars 82 forks source link

[BUG] Timestamp display arrow 13.0 #88

Closed MCRE-BE closed 10 months ago

MCRE-BE commented 10 months ago

Parquet Viewer Version What version of Parquet Viewer are you experiencing the issue with? 2.7.2.1

Where was the parquet file created? Apache Spark, Hive, Java, C#, pyarrow, etc.

Python pyarrow

Sample File Upload a sample file so the issue can be debugged!

Two sample dataframes with date features. Features.zip

Describe the bug A clear and concise description of what the bug is.

Screenshots

image image

Additional context Add any other context about the problem here.

Note: This tool relies on the parquet-dotnet library for all the actual Parquet processing. So any issues where that library cannot process a parquet file will not be addressed by us. Please open a ticket on that library's repo to address such issues.

mukunku commented 10 months ago

It looks like the Timestamp field is malformed in the Arrow 13 version as the ConvertedType is missing: bug

I added a patch to the app nonetheless so we can still open such fields and process them as DateTimes in v2.7.2.2

I also replied to your message on the PR: https://github.com/mukunku/ParquetViewer/pull/85#discussion_r1310506185

MCRE-BE commented 10 months ago

Thanks. I'll test the fix tomorrow.

MCRE-BE commented 10 months ago

Working. Thanks for the quick fix