[SEDONA-669] Fix timestamp_nz for GeoParquet reader and writer - Githubissues

apache / sedona

A cluster computing framework for processing large-scale geospatial data

https://sedona.apache.org/

Apache License 2.0

1.96k stars 692 forks source link

[SEDONA-669] Fix timestamp_nz for GeoParquet reader and writer #1661

Closed Kontinuation closed 3 weeks ago

Kontinuation commented 3 weeks ago

Did you read the Contributor Guide?

Yes, I have read the Contributor Rules and Contributor Development Guide

Is this PR related to a JIRA ticket?

Yes, the URL of the associated JIRA ticket is https://issues.apache.org/jira/browse/SEDONA-669. The PR name follows the format [SEDONA-XXX] my subject.

What changes were proposed in this PR?

Ports code for supporting the TimestampNTZ type added since Spark 3.4 to GeoParquet code, so that GeoParquet reader/writer won't throw exceptions when reading/writing parquet files containing timestamp_ntz columns.

How was this patch tested?

Added a new test case for testing the timestamp_ntz data type.
Tested on DBR 15.4 LTS to ensure no binary compatibility issues were introduced.

Did this PR include necessary documentation updates?

No, this PR does not affect any public API so no need to change the documentation.