search

vpc-ccg / pamir

Discovery and Genotyping of Novel Sequence Insertions in Many Sequenced Individuals
BSD 3-Clause "New" or "Revised" License
8 stars 4 forks source link

optimize computer usage #48

Open christinafliege opened 4 years ago

christinafliege commented 4 years ago

Good afternoon!

Pamir has been running successfully for many hours now. We have many more data sets we will be running it on, and so would like to optimize it and economically use our node hours.

Currently, we are using -j16 for our cluster configuration in the qsub script and running on a queue with 2 * Intel Xeon E5 2690 v3 CPUs (24 cores/node) and 256 GB of RAM. We would like to run on all 24 cores, would this be possible?

Additionally, we have access to other nodes which have 192GB and 40cores/node. So ideally we would like to be able to run on these nodes. Would that be possible?

fhach commented 4 years ago

Technically it can utilize more number of nodes. I will let @f0t1h and @mortunco help you on this matter.

fhach commented 4 years ago

Can you please post how you run pamir with your qsub.

mortunco commented 4 years ago

Dear @christinafliege

Are you able to share us how you run the pamir on your end. At this moment, all of ours servers are on slurm so we might ask you to run small example couple times to make it working.

Thank you for your help, T.

christinafliege commented 4 years ago

Here is the full qsub script. we run on pbs currently, Thank you!


#!/bin/bash

#PBS -N June19_real

#PBS -l nodes=1:ppn=24,walltime=168:00:00

#PBS -M cfliege2@illinois.edu

#PBS -m bae

#PBS -S /bin/bash

#PBS -A BANY

#PBS -q big_mem

cd /projects/mgc/Project_2/HLHS_BasilAniseVC/Pamir

module load pamir/pamir-2.1.0

 /usr/local/apps/bioapps/pamir/pamir-2.1.0/pamir.sh  -j16 --configfile /projects/mgc/Project_2/HLHS_BasilAniseVC/Pamir/realconfig.yaml
mortunco commented 4 years ago

@christinafliege oh we thought you were on SGE. okay we have access to PBS and finish this conversion asap.

thanks for your patience.