free-form description to incorporate into README.md or other publications
Work thus far
With recent advances in the coverage of high quality public domain aerial terrain imagery, civil designers, surface water modellers, and others can conduct a majority of their analyses directly from available terrain imagery before purchasing a more highly detailed land survey, if at all.
It is a common need and common practice today for practitioners to aggregate terrain imagery from a variety of aerial imagery sources, often received as datasets with differing spatial resolutions and map projections, and with differing and sometimes overlapping spatial coverage areas. In order to undertake their engineering analyses, it is usually necessary to use a single terrain dataset with the same spatial resolution and map projection throughout, covering the entire spatial extent of the area under analyses. For each area under analyses, practitioners conduct a similar series of pre-processing steps involving spatial reprojection, mosaicking, and retiling in order to produce a seamless terrain dataset with a single set of characteristics such as projection & resolution.
This is the order to scale these pre-processing steps beyond the local areas they are usually undertaken:
1) separate all available imagery tiles into collections by the following characteristics:
spatial projection
data type
color interpretation
release date
pixel resolution of imagery tiles
usual spacing of points in point clouds corresponding to each imagery tile
2) remove from consideration collections of tiles with non-float data types
3) modify color interpretation value to a single value for every tile in each collection
4) reproject tiles in each collection to the same spatial projection, both vertically and horizontally
5) mosaic tiles, using these two tables of elevation data in this order:
1) a table of source DEM tiles with a preference order determined by release date
2) a table of seamless DEM tiles with a preference order determined by their DEM resolutions
note: in both of these mosaics, secondary characteristics such as DEM resolution, LPC resolution, etc can be used to guarantee a consistent order of tiles, facilitating QAQC
6) select target characteristics of seamless terrain dataset to be:
single projection
floating point data type with enough bits to cover source datasets
single color interpretation
pixel resolution greater than any pixel resolution found in all available imagery, that is a positive or negative power of two
7) retile mosaic to these characteristics, producing a single seamless terrain dataset
This workflow is conducted on the imagery tile basis. There is an alternative workflow which is conducted on a pixel-by-pixel basis:
the target seamless terrain grid is established in advance, based in part on characteristics of all available imagery tiles
all steps before mosaicking & retiling can be conducted on a tile-by-tile basis, as above
mosaicking & retiling can be conducted on a pixel by pixel basis within a database table of those pixels, directly assigning pixel values using the preference order established above.
The key claim of this work is that spatial inaccuracies introduced by this generalized imagery tile aggregation workflow are minimal enough that the resulting data is as usable as the source data for the vast majority of analyses conducted. This claim relies upon the high quality (<3m) of the most recent generations of Lidar imagery and the most common analyses's tolerance for inaccuracy in the vast majority of their use-cases.
Each step of this aggregation workflow can be divided geographically into a computational grid, with each cell corresponding to a geographic extent that is a subset of the extent of all available imagery. Each cell can be separately computed in parallel on each CPU of each node.
Because the aggregation workflow can be computed independently for separate geographic extents, this same workflow can be applied to update sections of the final single seamless terrain tileset whenever new source data becomes available.
Vision
A web interface will serve this highest-resolution tileset. A separate and completely parallel (1-to-1) set of visualization tiles with reduced quality necessary for visual display over the Internet will be produced.
👉 This interface will include togglable semi-transparent layers describing visually on the map itself for each tile from the single seamless terrain dataset
which source dataset it came from,
the resolution of the source dataset,
the year of that source dataset,
etc.
👉 Another semi-transparent layer will describe the spatial inaccuracy of each pixel relative to its source dataset.
👉 Another semi-transparent layer will be a polygonal depiction of the best available source imagery tilesets at each point.
This interface will allow for search by but not limited to
county
watershed identification codes or plain names
neighborhood
parcel
vector file describing custom boundaries
This interface will provide zooming and panning of visualization tile map. This interface will provide modal selection of tiles by drawing:
rectangles
free-form polygons
This web interface will support downloading & receiving feedback for any selections above.
To support downloading these custom selections, aggregations of these tiles will be produced on request (on-the-fly) either for immediate download (if selection is small enough) or for later download by temporarily provisioned link.
A toggle will be provided to support downloading at different spatial resolutions, either fixed or custom. Since the single seamless terrain dataset is a maximally up-sampled terrain dataset representing the best available data at any given geographic location, all other spatial resolutions provided will be down-sampled from this single highest-resolution dataset.
This web interface will support on-the-fly requests to generate digital elevation tilesets for selections above directly from all available source Lidar point clouds, at a specified resolution and using a selected resampling algorithm.
free-form description to incorporate into
README.mdor other publicationsWork thus far
With recent advances in the coverage of high quality public domain aerial terrain imagery, civil designers, surface water modellers, and others can conduct a majority of their analyses directly from available terrain imagery before purchasing a more highly detailed land survey, if at all.
It is a common need and common practice today for practitioners to aggregate terrain imagery from a variety of aerial imagery sources, often received as datasets with differing spatial resolutions and map projections, and with differing and sometimes overlapping spatial coverage areas. In order to undertake their engineering analyses, it is usually necessary to use a single terrain dataset with the same spatial resolution and map projection throughout, covering the entire spatial extent of the area under analyses. For each area under analyses, practitioners conduct a similar series of pre-processing steps involving spatial reprojection, mosaicking, and retiling in order to produce a seamless terrain dataset with a single set of characteristics such as projection & resolution.
This is the order to scale these pre-processing steps beyond the local areas they are usually undertaken: 1) separate all available imagery tiles into collections by the following characteristics:
This workflow is conducted on the imagery tile basis. There is an alternative workflow which is conducted on a pixel-by-pixel basis:
The key claim of this work is that spatial inaccuracies introduced by this generalized imagery tile aggregation workflow are minimal enough that the resulting data is as usable as the source data for the vast majority of analyses conducted. This claim relies upon the high quality (<3m) of the most recent generations of Lidar imagery and the most common analyses's tolerance for inaccuracy in the vast majority of their use-cases.
Each step of this aggregation workflow can be divided geographically into a computational grid, with each cell corresponding to a geographic extent that is a subset of the extent of all available imagery. Each cell can be separately computed in parallel on each CPU of each node.
Because the aggregation workflow can be computed independently for separate geographic extents, this same workflow can be applied to update sections of the final single seamless terrain tileset whenever new source data becomes available.
Vision
A web interface will serve this highest-resolution tileset. A separate and completely parallel (1-to-1) set of visualization tiles with reduced quality necessary for visual display over the Internet will be produced.
👉 This interface will include togglable semi-transparent layers describing visually on the map itself for each tile from the single seamless terrain dataset
👉 Another semi-transparent layer will describe the spatial inaccuracy of each pixel relative to its source dataset.
👉 Another semi-transparent layer will be a polygonal depiction of the best available source imagery tilesets at each point.
This interface will allow for search by but not limited to
This web interface will support downloading & receiving feedback for any selections above.
To support downloading these custom selections, aggregations of these tiles will be produced on request (on-the-fly) either for immediate download (if selection is small enough) or for later download by temporarily provisioned link.
A toggle will be provided to support downloading at different spatial resolutions, either fixed or custom. Since the single seamless terrain dataset is a maximally up-sampled terrain dataset representing the best available data at any given geographic location, all other spatial resolutions provided will be down-sampled from this single highest-resolution dataset.
This web interface will support on-the-fly requests to generate digital elevation tilesets for selections above directly from all available source Lidar point clouds, at a specified resolution and using a selected resampling algorithm.