locationtech / geotrellis

GeoTrellis is a geographic data processing engine for high performance applications.
http://geotrellis.io
Other
1.34k stars 361 forks source link

How to Index Pyramid Tiles with Custom Encoding #3512

Open KiktMa opened 1 year ago

KiktMa commented 1 year ago
    Pyramid.upLevels(reprojected, tarlayoutScheme,startZoom, endZoom, Bilinear) { (rdd, z) =>
      val layerId = LayerId("layer_"+tableName, z)

      val tilesWithMetadata: RDD[(SpatialKey, Tile, TileLayerMetadata[String])] = rdd.map { case (key, tile) =>
        val topLeftCode = GeoSot.INSTANCE.getHexCode(rdd.metadata.extent.xmin, rdd.metadata.extent.ymax, 0, z)
        val bottomRightCode = GeoSot.INSTANCE.getHexCode(rdd.metadata.extent.xmax, rdd.metadata.extent.ymin, 0, z)
        val metaData = TileLayerMetadata(
          cellType = tile.cellType,
          layout = rdd.metadata.layout,
          extent = rdd.metadata.mapTransform(key),
          crs = rdd.metadata.crs,
          bounds = KeyBounds(topLeftCode, bottomRightCode)
        )
        (key, tile, metaData)
      }
      val indexKeyBounds: KeyBounds[SpatialKey] = {
        val KeyBounds(minKey, maxKey):KeyBounds[SpatialKey] = tilesWithMetadata.foreach((S) => S._1)
        KeyBounds(minKey, maxKey)
      }
      val keyIndex =
        ZCurveKeyIndexMethod.createIndex(indexKeyBounds)

      if (attributeStore.layerExists(layerId)) {
        AccumuloLayerDeleter(attributeStore).delete(layerId)
      }
      writer.write(layerId, rdd, keyIndex)
    }

Can someone help me solve this? I customized a tile encoding method, which obtains a string encoding based on the position information and hierarchy of each tile. I wanted to use this encoding to replace the encoding of the Z curve, but I reported an error during compilation,

[ERROR] D:\JavaConsist\Project\geotrellis-cuts\src\main\scala\com\spark\demo\index\SelfIndex.scala:113: type mismatch; found : Unit required: geotrellis.spark.KeyBounds[geotrellis.spark.SpatialKey] [ERROR] val KeyBounds(minKey, maxKey):KeyBounds[SpatialKey] = tilesWithMetadata.foreach((S) => S._ 1) [ERROR] ^ [WARNING] one warning found [ERROR] one error found

May I ask how to apply custom encoding to the pyramid model

pomadchin commented 1 year ago

Hey @KiktMa, sry got a bit busy and didn't pay attention much to the incoming issues, I'll look into the question asap!

pomadchin commented 1 year ago

Hmm, it is possible in this case to define a custom KeyIndexMethod and KeyIndex it wont be even neccessary writing a new code, all functions are generic enough to work with any KeyIndexMethod and KeyIndex definitions.

Could you descirbe on a bit more higher level, what's the idea behind these operations?

KiktMa commented 1 year ago

@pomadchin thank you for your reply I now need to use a geosot encoding to encode the grid. I already have a method to implement this encoding, but it needs to obtain the encoding based on any latitude and longitude of the grid and its level. Do I need to obtain latitude and longitude information based on SpatialKey (x, y)? How should I calculate it. I have another question to ask you. I am currently using writer. write(layerId, rdd, ZCurveKeyIndexMethod) to write tiles into Accumulo. Is it possible to store them in Accumulo and encode them as Z-curves? The rowid is like this \x00\x00\x00\x00\x00\x1E"\xCB, but I am a bit confused about what this is? If it is, I should also store them in rowid using geosot encoding.Finally, if I want to build tiles with a resolution of 0.5 meters at level 18, how can I do it? I use ZoomedLayoutScheme to build a pyramid with a resolution of around 0.29 meters at level 18. Sorry for raising so many questions. I hope you can reply when you have time. Thank you again