search

PolarGeospatialCenter / imagery_utils

Other
34 stars 10 forks source link

Slurm log location option #78

Closed bakkerbakker closed 5 months ago

bakkerbakker commented 5 months ago

Adds an option to pgc_ortho.py, pgc_pansharpen.py, and pgc_ndvi.py to set the output location for slurm logs when running operations on the cluster. This changes the default behavior of those scripts from the SLURM default of writing output/err logs in the working directory, instead making the default log output directory the parent directory of the script dst. So if the dst dir is project/imagery/ortho/ then the log from the cluster gets written to project/imagery/Or0001.oJOBID.

The change has not been applied to pgc_mosaic.py yet because that handles logging slightly differently.

This PR also fixes a bug from the AutoDEM PR: https://github.com/PolarGeospatialCenter/imagery_utils/pull/76 and applied a couple small fixes to pgc_ndvi.py to address some deprecation warnings for array handling.

bakkerbakker commented 5 months ago

NOTE: these changes have not been applied to the pgc_mosaic.py script because it handles subtile logging differently. This will be noted in a github issue. Slurm logging should now work as desired for Fridge processing (which does not utilize the mosaic scripts), which was the primary impetus for making these changes.