locationtech / geotrellis

GeoTrellis is a geographic data processing engine for high performance applications.
http://geotrellis.io
Other
1.33k stars 362 forks source link

FileCOGLayerWriter.write from SparkCOGExamples issue #3492

Open xvThomas opened 1 year ago

xvThomas commented 1 year ago

I desperately try to apply the example given in https://github.com/locationtech/geotrellis/blob/master/doc-examples/src/main/scala/geotrellis/doc/examples/spark/COGSparkExamples.scala, 'Having an RDD ...' with a srtm geotiff as follow:

   val conf = new SparkConf()
      .setMaster("local[*]")
      .setAppName("geotiff-tiler-spark")
      .set("spark.driver.host", "localhost")

    implicit val sc: SparkContext = new SparkContext(conf)

    val localGeoTiffPath =
      new java.io.File(new java.io.File(".").getCanonicalFile, "./data/srtm_39_03.tif").getAbsolutePath

    val localCatalogPath =
      new java.io.File(new java.io.File(".").getCanonicalFile, "./data/catalog").getAbsolutePath

    val rdd: RDD[(ProjectedExtent, Tile)] = HadoopGeoTiffRDD.spatial(new Path(localGeoTiffPath))

    val layoutScheme                                         = FloatingLayoutScheme(512)
    val (zoom: Int, metadata: TileLayerMetadata[SpatialKey]) = rdd.collectMetadata[SpatialKey](layoutScheme)
    val tilerOptions = Tiler.Options(resampleMethod = Bilinear, partitioner = Some(new HashPartitioner(rdd.partitions.length)))

    val tiledRdd                           = rdd.tileToLayout[SpatialKey](metadata, tilerOptions)
    val layerRdd: TileLayerRDD[SpatialKey] = ContextRDD(tiledRdd, metadata)

    val attributeStore = FileAttributeStore(localCatalogPath)
    val writer         = FileCOGLayerWriter(attributeStore)

    writer.write(layerName = "srtm_39_03", tiles = layerRdd, tileZoom = zoom, keyIndexMethod = ZCurveKeyIndexMethod)

But when the FileCOGLayerWriter write operation is executed, I got the following error:

Exception in thread "main" java.lang.RuntimeException: Tile Layout of layer and ZoomedLayoutScheme do not match. LayoutDefinition(Extent(10.0, 44.88000000000005, 15.120000000000015, 50.0),CellSize(8.333333333333358E-4,8.333333333333248E-4),12x12 tiles,6144x6144 pixels) != LayoutDefinition(Extent(-180.0, -89.99999, 179.99999, 89.99999),CellSize(0.70312498046875,0.3515624609375),1x1 tiles,512x512 pixels)

which corresponds to an assertion from COGLayer.scala:

if(rdd.metadata.layout != layoutScheme.levelForZoom(baseZoom).layout) {
    sys.error(s"Tile Layout of layer and ZoomedLayoutScheme do not match. ${rdd.metadata.layout} != ${layoutScheme.levelForZoom(baseZoom).layout}")
}

Any idea or explanation ?

For information, here is gdalinfo output of the srtm file:

gdalinfo srtm_39_03.tif
Driver: GTiff/GeoTIFF
Files: srtm_39_03.tif
Size is 6000, 6000
Coordinate System is:
GEOGCRS["WGS 84",
    ENSEMBLE["World Geodetic System 1984 ensemble",
        MEMBER["World Geodetic System 1984 (Transit)"],
        MEMBER["World Geodetic System 1984 (G730)"],
        MEMBER["World Geodetic System 1984 (G873)"],
        MEMBER["World Geodetic System 1984 (G1150)"],
        MEMBER["World Geodetic System 1984 (G1674)"],
        MEMBER["World Geodetic System 1984 (G1762)"],
        MEMBER["World Geodetic System 1984 (G2139)"],
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]],
        ENSEMBLEACCURACY[2.0]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    USAGE[
        SCOPE["Horizontal component of 3D system."],
        AREA["World."],
        BBOX[-90,-180,90,180]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Origin = (10.000000000000000,50.000000000000000)
Pixel Size = (0.000833333333333,-0.000833333333333)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  10.0000000,  50.0000000) ( 10d 0' 0.00"E, 50d 0' 0.00"N)
Lower Left  (  10.0000000,  45.0000000) ( 10d 0' 0.00"E, 45d 0' 0.00"N)
Upper Right (  15.0000000,  50.0000000) ( 15d 0' 0.00"E, 50d 0' 0.00"N)
Lower Right (  15.0000000,  45.0000000) ( 15d 0' 0.00"E, 45d 0' 0.00"N)
Center      (  12.5000000,  47.5000000) ( 12d30' 0.00"E, 47d30' 0.00"N)
Band 1 Block=6000x1 Type=Int16, ColorInterp=Gray
  NoData Value=-32768
pomadchin commented 1 year ago

Hey @xvThomas! I think the doc is a bit inaccurate; COOGLayer can be used with the ZoomedLayoutScheme only at this point;

see https://github.com/locationtech/geotrellis/issues/2609#issue-310211210

But it's a good catch - I totally forgot about the docs.