mukunku / ParquetViewer

Simple Windows desktop application for viewing & querying Apache Parquet files
GNU General Public License v3.0
785 stars 99 forks source link

[BUG] Timestamp display arrow 13.0 #88

Closed MCRE-BE closed 1 year ago

MCRE-BE commented 1 year ago

Parquet Viewer Version What version of Parquet Viewer are you experiencing the issue with? 2.7.2.1

Where was the parquet file created? Apache Spark, Hive, Java, C#, pyarrow, etc.

Python pyarrow

Sample File Upload a sample file so the issue can be debugged!

Two sample dataframes with date features. Features.zip

Describe the bug A clear and concise description of what the bug is.

Screenshots

image image

Additional context Add any other context about the problem here.

Note: This tool relies on the parquet-dotnet library for all the actual Parquet processing. So any issues where that library cannot process a parquet file will not be addressed by us. Please open a ticket on that library's repo to address such issues.

mukunku commented 1 year ago

It looks like the Timestamp field is malformed in the Arrow 13 version as the ConvertedType is missing: bug

I added a patch to the app nonetheless so we can still open such fields and process them as DateTimes in v2.7.2.2

I also replied to your message on the PR: https://github.com/mukunku/ParquetViewer/pull/85#discussion_r1310506185

MCRE-BE commented 1 year ago

Thanks. I'll test the fix tomorrow.

MCRE-BE commented 1 year ago

Working. Thanks for the quick fix