The base assumption in Virtual Layer Mosaic is that we're working in RDD context where the work of querying, filtering and reading can be distributed.
However, in this use case we're interested in producing a ValueReader that we can query from async service to produce tiles from the virtual layer. This means that the virtual layer needs to be either collected to the driver or preferably have an ability to be constructed outside of SparkContext.
This would support application like raster-foundry that by design have UI driven ways to assemble a mosaic that needs to be consumed through a tile server.
Thus the expectation is that this ValueReader instance will have to handle high number of concurrent tile requests.
The main performance concerns are:
The need for spatial index to efficiently select source rasters for target tile.
The need for pluggable cache to either store the raster headers to avoid re-reading it per request or to amortize header reading over NxM tile window as a pre-cache strategy.
Both in-memory, on-disk, and over-network caches are valid here depending on application.
This idea has been explored in context of "unstructured COG layers" here.
That implementation should be reconciled with Virtual Layer Mosaic.
Connects: https://github.com/locationtech/geotrellis/issues/2698
The base assumption in Virtual Layer Mosaic is that we're working in RDD context where the work of querying, filtering and reading can be distributed.
However, in this use case we're interested in producing a
ValueReader
that we can query from async service to produce tiles from the virtual layer. This means that the virtual layer needs to be either collected to the driver or preferably have an ability to be constructed outside ofSparkContext
.This would support application like
raster-foundry
that by design have UI driven ways to assemble a mosaic that needs to be consumed through a tile server. Thus the expectation is that thisValueReader
instance will have to handle high number of concurrent tile requests.The main performance concerns are:
This idea has been explored in context of "unstructured COG layers" here. That implementation should be reconciled with Virtual Layer Mosaic.