locationtech / geotrellis

GeoTrellis is a geographic data processing engine for high performance applications.
http://geotrellis.io
Other
1.34k stars 361 forks source link

Tile Server for Virtual Layer Mosaic #2703

Open echeipesh opened 6 years ago

echeipesh commented 6 years ago

Connects: https://github.com/locationtech/geotrellis/issues/2698

The base assumption in Virtual Layer Mosaic is that we're working in RDD context where the work of querying, filtering and reading can be distributed.

However, in this use case we're interested in producing a ValueReader that we can query from async service to produce tiles from the virtual layer. This means that the virtual layer needs to be either collected to the driver or preferably have an ability to be constructed outside of SparkContext.

This would support application like raster-foundry that by design have UI driven ways to assemble a mosaic that needs to be consumed through a tile server. Thus the expectation is that this ValueReader instance will have to handle high number of concurrent tile requests.

The main performance concerns are:

  1. The need for spatial index to efficiently select source rasters for target tile.
  2. The need for pluggable cache to either store the raster headers to avoid re-reading it per request or to amortize header reading over NxM tile window as a pre-cache strategy.
    • Both in-memory, on-disk, and over-network caches are valid here depending on application.

This idea has been explored in context of "unstructured COG layers" here. That implementation should be reconciled with Virtual Layer Mosaic.

pomadchin commented 5 years ago

https://github.com/geotrellis/geotrellis-server depends on VLM at the moment.