search

locationtech / geotrellis

GeoTrellis is a geographic data processing engine for high performance applications.
http://geotrellis.io
Other
1.32k stars 360 forks source link

IndexOutOfBoundsException occurs when reading tiff #3509

Open imperio-wxm opened 1 year ago

imperio-wxm commented 1 year ago

Describe the bug

I get this error when reading tiff.

23/03/30 14:22:20 ERROR Executor: Exception in task 0.0 in stage 3.0 (TID 3)
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:567)
at java.nio.ByteBufferAsShortBufferB.get(ByteBufferAsShortBufferB.java:114)
at geotrellis.raster.io.geotiff.UInt16GeoTiffSegment.getRaw(UInt16GeoTiffSegment.scala:32)
at geotrellis.raster.io.geotiff.UInt16ConstantNoDataGeoTiffSegment.getInt(UInt16ConstantNoDataGeoTiffSegment.scala:25)
at geotrellis.raster.io.geotiff.GeoTiffMultibandTile.$anonfun$crop$7(GeoTiffMultibandTile.scala:584)
at geotrellis.raster.io.geotiff.GeoTiffMultibandTile.$anonfun$crop$7$adapted(GeoTiffMultibandTile.scala:566)
at scala.collection.immutable.List.foreach(List.scala:431)
at geotrellis.raster.io.geotiff.GeoTiffMultibandTile.burnPixelInterleave$1(GeoTiffMultibandTile.scala:566)
at geotrellis.raster.io.geotiff.GeoTiffMultibandTile.$anonfun$crop$10(GeoTiffMultibandTile.scala:639)
at scala.collection.Iterator$$anon$11.nextCur(Iterator.scala:486)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:492)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
at geotrellis.raster.geotiff.GeoTiffResampleRasterSource.read(GeoTiffResampleRasterSource.scala:96)
at geotrellis.raster.RasterSource.read(RasterSource.scala:128)
at geotrellis.raster.RasterRegion$GridBoundsRasterRegion.$anonfun$raster$1(RasterRegion.scala:44)
at scala.Option.flatMap(Option.scala:271)
at geotrellis.raster.RasterRegion$GridBoundsRasterRegion.raster$lzycompute(RasterRegion.scala:43)
at geotrellis.raster.RasterRegion$GridBoundsRasterRegion.raster(RasterRegion.scala:41)
at geotrellis.spark.RasterSourceRDD$.$anonfun$tiledLayerRDD$12(RasterSourceRDD.scala:276)
at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
at scala.collection.Iterator.foreach(Iterator.scala:943)
at scala.collection.Iterator.foreach$(Iterator.scala:943)
at org.apache.spark.util.collection.CompactBuffer$$anon$1.foreach(CompactBuffer.scala:115)
at scala.collection.IterableLike.foreach(IterableLike.scala:74)
at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
at org.apache.spark.util.collection.CompactBuffer.foreach(CompactBuffer.scala:32)
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
at org.apache.spark.util.collection.CompactBuffer.flatMap(CompactBuffer.scala:32)
at geotrellis.spark.RasterSourceRDD$.$anonfun$tiledLayerRDD$11(RasterSourceRDD.scala:276)
at org.apache.spark.rdd.PairRDDFunctions.$anonfun$mapValues$3(PairRDDFunctions.scala:751)
at scala.collection.Iterator$$anon$10.next(Iterator.scala:461)
at org.apache.spark.util.Utils$.getIteratorSize(Utils.scala:1891)
at org.apache.spark.rdd.RDD.$anonfun$count$1(RDD.scala:1253)
at org.apache.spark.rdd.RDD.$anonfun$count$1$adapted(RDD.scala:1253)
at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2254)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:131)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

To Reproduce

Tiff: cog.tif.zip

Code:

val path = "./cog.tif"

val sourceRDD: RDD[RasterSource] = sc.parallelize(Seq(path)).map(uri => {
  GeoTiffRasterSource(uri): RasterSource
})

val summary = RasterSummary.fromRDD(sourceRDD)
val LayoutLevel(zoom, layout) = summary.levelFor(FloatingLayoutScheme(256))
val contextRDD = RasterSourceRDD.tiledLayerRDD(sourceRDD, layout, KeyExtractor.spatialKeyExtractor, rasterSummary = Some(summary))
contextRDD.count()

Gdal Info:

Driver: GTiff/GeoTIFF
Files: cog.tif
Size is 7611, 7751
Coordinate System is:
PROJCRS["WGS 84 / UTM zone 50N",
    BASEGEOGCRS["WGS 84",
        DATUM["World Geodetic System 1984",
            ELLIPSOID["WGS 84",6378137,298.257223563,
                LENGTHUNIT["metre",1]]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4326]],
    CONVERSION["UTM zone 50N",
        METHOD["Transverse Mercator",
            ID["EPSG",9807]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",117,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["Scale factor at natural origin",0.9996,
            SCALEUNIT["unity",1],
            ID["EPSG",8805]],
        PARAMETER["False easting",500000,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]]],
    CS[Cartesian,2],
        AXIS["(E)",east,
            ORDER[1],
            LENGTHUNIT["metre",1]],
        AXIS["(N)",north,
            ORDER[2],
            LENGTHUNIT["metre",1]],
    USAGE[
        SCOPE["unknown"],
        AREA["World - N hemisphere - 114°E to 120°E - by country"],
        BBOX[0,114,84,120]],
    ID["EPSG",32650]]
Data axis to CRS axis mapping: 1,2
Origin = (552885.000000000000000,4264815.000000000000000)
Pixel Size = (30.000000000000000,-30.000000000000000)
Metadata:
  AREA_OR_POINT=Point
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  552885.000, 4264815.000) (117d36'24.28"E, 38d31'48.59"N)
Lower Left  (  552885.000, 4032285.000) (117d35'24.10"E, 36d26' 3.57"N)
Upper Right (  781215.000, 4264815.000) (120d13'26.65"E, 38d29'14.41"N)
Lower Right (  781215.000, 4032285.000) (120d 8' 7.50"E, 36d23'40.56"N)
Center      (  667050.000, 4148550.000) (118d53'20.55"E, 37d28' 7.50"N)
Band 1 Block=512x512 Type=UInt16, ColorInterp=Gray
  Overviews: 3806x3876, 1903x1938, 952x969, 476x485, 238x243, 119x122, 60x61

Environment