PolarGeospatialCenter / imagery_utils

Other
34 stars 10 forks source link

Add capability to ortho with `--dem` directory containing DEM mosaic tile dataset (WIP) #67

Closed ehusby closed 5 months ago

ehusby commented 1 year ago

Currently these are untested changes adding new arguments to pgc_ortho.py and pgc_pansharpen.py and a lib/ortho_functions.preproc_demdir() function to be run in the head script before ortho task submission.

Associated User Services Trello card:

The currently undocumented capability pgc_ortho.py has for working on a source CSV "argument list" textfile is described briefly in the code here: https://github.com/PolarGeospatialCenter/imagery_utils/blob/4081ea63f5ae8427d115094c48176485f0de22d6/lib/utils.py#L742

The CSV file should have a src/O_FILEPATH column and a dem column that has the path to the DEM mosaic tile associated with the src image. Note that the spatial join that needs to be done to produce these columns should be a one-to-many join of PGC Imagery MFP geometries to the DEM mosaic tile geometries. This allows the CSV to contain information on all DEM tiles that each image overlaps.

When providing a CSV argument list textfile to pgc_ortho.py for this purpose, the --dem argument should be the *.vrt VRT file that was built with all of the DEM mosaic tiles, and that the mosaic tile paths it was built with are the same paths that are listed in the dem column of the CSV argument list textfile.

There is a corresponding Trello card on the PGC Developer board that goes into greater detail on the steps to automate this capability: https://trello.com/c/PAdWxebc/20-further-automate-ortho-by-dem-tileset-capability

Associated Developer Trello card:

Automate the workflow process needed to orthorectify a large set of imagery with a DEM tileset (such as TanDEM-X tiles).

The core of the process has already been implemented in PR, "Subset VRT tile mosaic DEM argument using src CSV argument list": https://github.com/PolarGeospatialCenter/imagery_utils/pull/39

Modify pgc_ortho.py (and pgc_pansharpen.py, or merge the two scripts at some point -- https://github.com/PolarGeospatialCenter/imagery_utils/issues/23) to identify when and how these added automation steps will kick in.

First level: If the --dem argument is a Shapefile or {VRT file, and a new flag such as --subset-tiled-dem is provided}, follow the Automation Steps below.

Second level: If the --dem argument is a directory containing DEM raster tiles, create both the Shapefile and VRT representations of the tileset, then follow Automation Steps.

Automation Steps:

  1. Produce VRT and/or Shapefile representations of the DEM tileset if not already available, given one of the two is provided as the --dem argument (or create a different input script argument if that makes sense), OR given an input path to the top-level directory that contains the DEM tileset. First, maybe you only want to accept either a VRT or Shapefile input, and assume that the the complement has already been produced (probably located right next to one another in the filesystem).
  2. Footprint source imagery
  3. Spatial join imagery footprint to DEM tile index (one-to-many join)
  4. Export spatial join result as CSV file in format accepted as a source argument list by pgc_ortho.py (see PR referenced in above Description)
  5. Add full documentation for new capabilities to workflow docs
bakkerbakker commented 6 months ago

@SAliHossaini Here is a different approach for batch DEM selection in pgc_ortho that Erik was previously working on. The solution that we talked about is a different way of doing it than Erik's method here, which requires a spatial join of imagery footprints to a set of DEM tiles, but it might be worth reviewing as you implement the auto-dem approach detailed in the issue here: https://github.com/PolarGeospatialCenter/imagery_utils/issues/72