avoid using `/tmp` on compute

[philipmac]

for each compute env (dev/qa/prod) we should change default TMPDIR justification - public /tmp gets full, is not reliable.

something simple like:

if [[ -O /home/$USER/tmp && -d /home/$USER/tmp ]]; then
        TMPDIR=/home/$USER/tmp
else
        # You may wish to remove this line, it is there in case
        # a user has put a file 'tmp' in there directory or a
        rm -rf /home/$USER/tmp 2> /dev/null
        mkdir -p /home/$USER/tmp
        TMPDIR=$(mktemp -d /home/$USER/tmp/XXXX)
fi

TMP=$TMPDIR
TEMP=$TMPDIR

export TMPDIR TMP TEMP

would avoid these kinds of errors

https://serverfault.com/questions/72955/how-to-change-default-tmp-to-home-user-tmp

[philipmac]

consider using SlurmCluster(job_script_prologue)