As PhD members in Statistics, we have two types of ITSC source.
University level: Everyone owns 10 running jobs without limitation of cores simutanously.
Division level: Members from Statistic division could run 30 cores (or 30 jobs) at most.
Sometimes, when you have multiple tests in parallel like 100. Each one will take more than 1 hour. The wise thought is to use both resource of University and Division level.
Here comes the way: we want to use bash file to allocate first 10 jobs to University level and the rest for STAT.
The bash file is:
#!/bin/bash
dir="/lustre/project/Stat/path/of/names/of/genes"
cd /lustre/project/Stat/code/job/path/
num=1
c=11
for i in $(ls $dir)
do
if (($num < $c))
then
sbatch --export=p=${i} try3.job
echo $num
else
sbatch -p stat -q stat --export=p=${i} try3.job
fi
pwd
num=$[$num+1]
sleep 1
done
In this way, we can pass the gene name to job file, and the corresponding job file is:
As PhD members in Statistics, we have two types of ITSC source.
Sometimes, when you have multiple tests in parallel like 100. Each one will take more than 1 hour. The wise thought is to use both resource of University and Division level.
Here comes the way: we want to use bash file to allocate first 10 jobs to University level and the rest for STAT. The bash file is:
In this way, we can pass the gene name to job file, and the corresponding job file is:
We have pass the name to python. The rest thing you need to do :
In such way, we can run the same python script via system environment. Amazing!