Add support for any square layout level (not just from ZoomedLayoutScheme)

[JonMcPherson]

Overview

This PR changes the Options to be more general and accept any Seq[LayoutLevel] for the desired zoom levels. The only constraints being that each level must define a unique zoom, and the tile layout for each level must be square tileCols == tileRows. This allows VectorPipe to be used for different layout schemes other than the power-of-2 ZoomedLayoutScheme. One example use case would be for generating tiles matching the power-of-20 OpenLocationCode grid (my use case 😄).

The current Options() constructor was moved to a separate apply method which generates the layoutLevels using ZoomedLayoutScheme in the same way as before. So this change will probably not break any users BUT note that it will break for users that invoke the new Options() constructor directly rather than using the generated apply method (which is probably nobody?).

Demo

I tested this for my use case using the code below which I did not include in this PR (in PipelineSpec) because it is NOT a good unit test and was only used to prove that VectorPipe can generate tiles for other layout schemes AND for any layout extent (not just worldExtent)

However, this test does fail on only a couple tiles (out of hundreds) that were generated for the area extent but NOT the world extent. I am looking into exactly why this these tiles did not get generated when using the full worldExtent, but I figured the results looked good enough, and that it was probably something wrong with my code or something to do with rounding.

it("should generate for any square layout level") {
  // zoom       |0  |1   |2    |3      |4       |5
  // layoutCols |18 |360 |7200 |144000 |2880000 |57600000
  // layoutRows |9  |180 |3600 |72000  |1440000 |28800000

  val zoom = 3 // 0+
  val tileSize = 278 // cell size: 1m² at the equator, .5x1m at ±60° Latitude

  val zoomMultiplier = Math.pow(20, zoom - 1).toInt
  val layoutCols = 360 * zoomMultiplier
  val layoutRows = 180 * zoomMultiplier

  def run(extent: Extent, tileLayout: TileLayout, outputDir: Path): Unit = {
    val layoutLevel = LayoutLevel(zoom, LayoutDefinition(extent, tileLayout))
    val options = VectorPipe.Options(Seq(layoutLevel), LatLng, LatLng, useCaching = false, orderAreas = false)
    val pipeline = LayerTestPipeline("geom", outputDir.toUri)
    VectorPipe(wayGeoms, pipeline, options)
  }

  // extent around Isle of Man as OLC level 1 (9C6Q0000+) with tiles at OLC level 3
  val areaExtent = Extent(-5, 54, -4, 55)
  // (level3 / level1) layoutCols / 360 == layoutRows / 180 == 400
  val areaExtentTileLayout = TileLayout(400, 400, tileSize, tileSize)
  val areaExtentTileDir = Files.createTempDirectory("olc-tiles-area-extent")
  run(areaExtent, areaExtentTileLayout, areaExtentTileDir)

  import geotrellis.layer._ // for worldExtent implicit
  val worldExtent = LatLng.worldExtent
  val worldExtentTileLayout = TileLayout(layoutCols, layoutRows, tileSize, tileSize)
  val worldExtentTileDir = Files.createTempDirectory("olc-tiles-world-extent")
  run(worldExtent, worldExtentTileLayout, worldExtentTileDir)

  println(s"areaExtentTileDir: $areaExtentTileDir")
  println(s"worldExtentTileDir: $worldExtentTileDir")

  val tileSizeDegrees = BigDecimal(360) / layoutCols
  val worldExtentTileOffsetX = (layoutCols / 2) + (areaExtent.xmin * areaExtentTileLayout.layoutCols) // 70000
  val worldExtentTileOffsetY = (layoutRows / 2) - (areaExtent.ymax * areaExtentTileLayout.layoutRows) // 14000

  Files.list(areaExtentTileDir.resolve(zoom.toString)).iterator().asScala.foreach { areaExtentTileColPath =>
    val areaExtentTileX = areaExtentTileColPath.getFileName.toString.toInt
    val worldExtentTileX = worldExtentTileOffsetX + areaExtentTileX

    Files.list(areaExtentTileColPath).iterator().asScala.filter(_.toString.endsWith(".mvt")).foreach { areaExtentTilePath =>
      val areaExtentTileY = areaExtentTilePath.getFileName.toString.replace(".mvt", "").toInt
      val worldExtentTileY = worldExtentTileOffsetY + areaExtentTileY

      val worldExtentTilePath = worldExtentTileDir
        .resolve(zoom.toString)
        .resolve(worldExtentTileX.toInt.toString)
        .resolve(s"${worldExtentTileY.toInt}.mvt")

      println(worldExtentTilePath)
      assert(Files.exists(worldExtentTilePath), worldExtentTilePath)

      //val xmin = ((tileSizeDegrees * worldExtentTileX) - 180).toDouble
      //val ymin = -((tileSizeDegrees * worldExtentTileY) - 90).toDouble
      //val tileExtent = Extent(xmin, ymin, (xmin + tileSizeDegrees).toDouble, (ymin + tileSizeDegrees).toDouble)
      //val areaExtentTile = VectorTile.fromBytes(Files.readAllBytes(areaExtentTilePath), tileExtent)
      //val worldExtentTile = VectorTile.fromBytes(Files.readAllBytes(worldExtentTilePath), tileExtent)
    }
  }
}

Testing Instructions

Existing unit tests aren't complete and don't assert anything, but they can still be run and the results manually inspected it seems.

Checklist

• [ ] Add entry to CHANGELOG.md

Closes #XXX

[jpolchlo]

Will give this a look soon. Thanks for the contribution!

[jpolchlo]

Just as a follow-up, please update the changelog!

[JonMcPherson]

@jpolchlo oh right, it is done.

[jpolchlo]

Great, thanks. I'll merge this into master, but I think according to semver, we're going to have to bump to 3.0 to release. We'll discuss on our end timing for that release, in case there are any other breaking changes that are warranted (i.e., it might not be right away).

Thanks for your work!