nsidc / qgreenland

Source code for generating the QGreenland package hosted at https://qgreenland.org/
https://qgreenland.readthedocs.io
Other
36 stars 9 forks source link

Experiment with GDAL performance options #627

Open MattF-NSIDC opened 1 year ago

MattF-NSIDC commented 1 year ago

https://kokoalberti.com/articles/geotiff-compression-optimization-guide/

GDAL_DISABLE_READDIR_ON_OPEN

When GDAL opens a file from a local directory or network, it will automatically try and establish a list of other files present in that location. This can result in a delay when accessing files over a network, or in a local directory with more than a few hundred other files. The behaviour can be prevented by setting the configuration option GDAL_DISABLE_READDIR_ON_OPEN to TRUE.

GDAL_CACHEMAX

The GDAL_CACHEMAX configuration option can be used to set the raster block size cache. In some quick tests I did not see any performance improvements in read or write speeds. Perhaps this was because the default values are already large enough in later versions of GDAL, or that I was not working with extremely large files where caching would be required. Others reported much faster read times on large files, so it may be worth experimenting with. You can set it to 1000MB using --config GDAL_CACHEMAX 1000 in your GDAL command.