mukunku / ParquetViewer

Simple Windows desktop application for viewing & querying Apache Parquet files
GNU General Public License v3.0
754 stars 91 forks source link

[FEATURE REQUEST] Display timestamp fields in human-intelligible format #40

Closed san-r closed 3 years ago

san-r commented 3 years ago

ParquetViewer is the best parquet file viewer. However, it shows 'timestamp' fields in Unix epoch (or Unix time or POSIX time or Unix timestamp), i.e., the number of seconds that have elapsed since January 1, 1970 (midnight UTC/GMT). For example,

image

The 'date' field is displayed just fine. It is requested that 'date-time' field be made viewable in human-intelligible format.

Meanwhile, I've posted a request at https://stackoverflow.com/questions/68741520/date-time-timestamp-field-in-parquet-file-shown-as-numbers-in-parquet-file-vie to see if something (like formatting change) could be done in R environment before exporting to parquet format so that it displays 'timestamp' field in human-intelligible format in ParquetViewer.

mukunku commented 3 years ago

Can you share a sample parquet file? This should be doable

san-r commented 3 years ago

Sure, here is the same parquet file shown in the screenshot in original post: testFyl.zip

This time I'm posting from a different machine where it looks like this:

image

Thanks for your efforts.

mukunku commented 3 years ago

Give this a shot and let me know if it works for you: ParquetViewer.zip

I compared the values against https://www.online-toolz.com/tools/unix-timestamp-to-date.php and it looks okay to me. (Keep in mind this website converts to your local timezone whilst in Parquet Viewer it will remain as UTC)

san-r commented 3 years ago

Excellent job. This version shows the timestamps properly. But I must apologize for providing an inadequate sample file which had time only upto hours and minutes and omitted out seconds. It is enough for displaying in ParquetViewer in almost all cases.

But just in case it is needed, here is a second sample parquet file which has seconds also: sample2.zip

The above version of ParquetViewer shows it as:

image

While in RStudio, where I created the sample file, it shows the seconds also:

image

I should reiterate that the current version shows timestamps properly and is adequate for me. But I thought that I should inform anyway.

Thanks again for your efforts.

mukunku commented 3 years ago

Thanks for being so helpful. I took a look and I think the default formatting for dates doesn't include the seconds. If you switch the date format to ISO 8601 you can see the seconds: image

Closing out this ticket.