search

uabrc / devops-docs

https://docs.rc.uab.edu/devops-docs/
Apache License 2.0
1 stars 7 forks source link

Scripts for controlling job times sbatch #35

Open wwarriner opened 1 year ago

wwarriner commented 1 year ago 
function slurm-job-add-timelimit() {
    timelimit="$1"
    jobids=("${@:2}")

    for job in "${jobids[@]}"; do
      echo "$job"
      scontrol show job $job | grep TimeLimit | awk '{print "  Before: " $2}'
      scontrol update jobid=$job TimeLimit+="$timelimit"
      scontrol show job $job | grep TimeLimit | awk '{print "  After:  " $2}'
    done
}

function slurm-job-check-timelimit() {
    jobids=("${@:1}")

    for job in "${jobids[@]}"; do
      echo "$job"
      scontrol show job $job | grep TimeLimit | awk '{print "  Time Limit: " $2}'
    done
}

The following code could be useful in niche usecases, but it can also force a job to terminate if the input timelimit is shorter than the currently elapsed time.

function slurm-job-set-timelimit() {
    timelimit="$1"
    jobids=("${@:2}")

    for job in "${jobids[@]}"; do
      echo "$job"
      scontrol show job $job | grep TimeLimit | awk '{print "  Before: " $2}'
      scontrol update jobid=$job TimeLimit="$timelimit"
      scontrol show job $job | grep TimeLimit | awk '{print "  After:  " $2}'
    done
}