UU-Hydro / PCR-GLOBWB_model

PCR-GLOBWB (PCRaster Global Water Balance) is a large-scale hydrological model intended for global to regional studies and developed at the Department of Physical Geography, Utrecht University (Netherlands). Contact: Edwin Sutanudjaja (E.H.Sutanudjaja@uu.nl).
GNU General Public License v3.0
116 stars 79 forks source link

Continue simulation and signal dump states features #41

Closed BramDr closed 4 months ago

BramDr commented 5 months ago

This request adds two new features to PCR-GLOBWB and closes issue #39 and closes issue #40 . Both features have been tested.

  1. A deterministic runner argument "-continue-previous" that will continue a previous simulation from the latest state file (taking into account the need to start at the start of the year or month when yearly or monthly outputs are requested).
  2. A deterministic runner argument "-dump-signal [signal-name]" that will cause the simulation to dump its state (after the current timestep) and exit when a system signal is received. Default signal is SIGUSR1, but others can be specified after the argument.

The functionality these features provide are very useful for long-running simulations that may be timed-out by the slurm scheduler. Using these options will prevent rerunning the whole simulation every time it stops OR splitting long-running simulations into periods.

Not only can you always restart the simulation from the last year (as PCR-GLOBWB will dump its states every year), but you can also ask slurm to send a signal to its processes a certain time before the timeout (#SBATCH --signal=[{R|B}:][@sig_time]; see https://slurm.schedmd.com/sbatch.html).

Cheers from me and Barry!