[SEDONA-646] Fix DBR compatibility problem for shapefile reader - Githubissues

apache / sedona

A cluster computing framework for processing large-scale geospatial data

https://sedona.apache.org/

Apache License 2.0

1.88k stars 662 forks source link

[SEDONA-646] Fix DBR compatibility problem for shapefile reader #1573

Closed Kontinuation closed 4 weeks ago

Kontinuation commented 4 weeks ago

Did you read the Contributor Guide?

Yes, I have read the Contributor Rules and Contributor Development Guide

Is this PR related to a JIRA ticket?

Yes, the URL of the associated JIRA ticket is https://issues.apache.org/jira/browse/SEDONA-646. The PR name follows the format [SEDONA-XXX] my subject.

What changes were proposed in this PR?

org.apache.spark.sql.execution.datasources.FilePartition on DBR is not binary compatible with the open-source Apache Spark, so we define our own InputPartition subclass to get rid of this problem.

How was this patch tested?

Manually tested on DBR 15.4 LTS.

Did this PR include necessary documentation updates?

No, this PR does not affect any public API so no need to change the documentation.