dtarb / TauDEM

Terrain Analysis Using Digital Elevation Models (TauDEM) software for hydrologic terrain analysis and channel network extraction.
http://hydrology.usu.edu/taudem
Other
222 stars 115 forks source link

Implement multifile VRT parallel raster output #241

Open jcphill opened 1 year ago

jcphill commented 1 year ago

Greatly improve raster output performance for large runs on parallel filesystems by writing one TIF file per rank along with a VRT file. Writing to /path/to/foldername/filename.vrt creates the directory foldername if it does not exist and writes data to files /path/to/foldername/filename.%d.tif

No changes are needed to read VRT files.

The VRT TIF files are written in tiled format as there is no performance penalty when writing but a dramatic performance increase when reading small areas from very wide rasters.