New --threads option that works with pgc_ortho.py and pgc_pansharpen.py to accelerate the gdalwarp and/or gdal_pansharpen.py processing. The flag accepts any nonzero integer or ALL_CPUS as arguments. Parser checks are in place to disable this in --pbs/--slurm processing, and will not let (threads * parallel processes) exceed the number of threads available on the system. Any standalone --threads argument exceeding the system thread count will default to the maximum available.
For gdalwarp this was implemented in the --config and warp options -wo flags. My testing found that threading of input dataset read -oo and output dataset write -doo had no effect. The requestor also tested these settings and found they improved performance.
For gdal_pansharpen.py the -threads flag was added. When --threads is supplied to pgc_pansharpen.py threading will occur for both the gdalwarp and gdal_pansharpen.py processes.
The README has been updated with the new flag. A bugfix for a print string error is also included.
ehusby
commented
3 years ago
New
--threadsoption that works withpgc_ortho.pyandpgc_pansharpen.pyto accelerate the gdalwarp and/or gdal_pansharpen.py processing. The flag accepts any nonzero integer or ALL_CPUS as arguments. Parser checks are in place to disable this in--pbs/--slurmprocessing, and will not let (threads * parallel processes) exceed the number of threads available on the system. Any standalone--threadsargument exceeding the system thread count will default to the maximum available.For
gdalwarpthis was implemented in the--configand warp options-woflags. My testing found that threading of input dataset read-ooand output dataset write-doohad no effect. The requestor also tested these settings and found they improved performance.For
gdal_pansharpen.pythe-threadsflag was added. When--threadsis supplied topgc_pansharpen.pythreading will occur for both thegdalwarpandgdal_pansharpen.pyprocesses.The README has been updated with the new flag. A bugfix for a print string error is also included.