mraad / FileGDB

Read GDB (Esri File GeoDatabase) using Apache Spark
Apache License 2.0
14 stars 6 forks source link

Fixed the problem that the coordinate returned is null when Z value or M value are NaN #4

Closed dingsl-giser closed 2 years ago

dingsl-giser commented 2 years ago

What changes were proposed in this pull request?

Fixed the problem that the coordinate returned is null when Z value or M value are NaN.

Why are the changes needed?

When Z or M in Geometry is NaN, the program returns null with an error message.

The program returns the result

[0,null,0.0,0.0,0.0,0.0]

The program returns an error message

java.nio.BufferUnderflowException
    at java.base/java.nio.Buffer.nextGetIndex(Buffer.java:643)
    at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:165)
    at com.esri.gdb.package$ByteBufferImplicits.getVarInt(package.scala:50)
    at com.esri.gdb.FieldMultiPartZM.readValue(FieldMultiPartZM.scala:83)
    at com.esri.gdb.FieldMultiPartZM.readValue(FieldMultiPartZM.scala:8)
    at com.esri.gdb.GDBTableIterator.$anonfun$next$1(GDBTableIterator.scala:39)
    at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
    at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
    at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
    at scala.collection.TraversableLike.map(TraversableLike.scala:238)
    at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
    at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:198)
    at com.esri.gdb.GDBTableIterator.next(GDBTableIterator.scala:33)
    at com.esri.gdb.GDBTableIterator.next(GDBTableIterator.scala:10)
    at scala.collection.Iterator$$anon$10.next(Iterator.scala:459)
    at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
    at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
    at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:729)
    at org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:340)
    at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:872)
    at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:872)
    at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:349)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:313)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:127)
    at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:446)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1377)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:449)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)

View the coordinate information in Pro:

Z-M Value For Pro

Returns the correct result with the updated code.

[3,[116.62030080573766,34.69734564108853,116.62060905556352,34.69757354278484,WrappedArray(7, 6),WrappedArray(116.62048341937441, 34.69757354278482, 0.0, NaN, 116.62060613456549, 34.69753263712158, 0.0, NaN, 116.62060905556348, 34.69740699913376, 0.0, NaN, 116.62042059993075, 34.69734564108853, 0.0, NaN, 116.62034609379725, 34.69739238964712, 0.0, NaN, 116.62030080573766, 34.697506340944926, 0.0, NaN, 116.62048341937441, 34.69757354278482, 0.0, NaN, 116.62048195797604, 34.69750780234324, 0.0, NaN, 116.62038845995949, 34.6974712790763, 0.0, NaN, 116.62046004419574, 34.6974259910167, 0.0, NaN, 116.62053162843199, 34.69743183481137, 0.0, NaN, 116.62053601082835, 34.69750488044593, 0.0, NaN, 116.62048195797604, 34.69750780234324, 0.0, NaN)],0.0012426,0.00124259990248,0.0012425999024696188,4.0812036749474E-8]