nservant / HiC-Pro

HiC-Pro: An optimized and flexible pipeline for Hi-C data processing
Other
372 stars 181 forks source link

Can multiple nodes be set for parallel processing in HiC-Pro (Torque)? #616

Closed YCMaCY closed 5 months ago

YCMaCY commented 5 months ago

cat HiCPro_step1_hic_mouse.sh

!/bin/bash

PBS -l nodes=10:ppn=32,mem=60gb,walltime=

PBS -M

PBS -m ae

PBS -j eo

PBS -N HiCpro_hic_mouse_pac

PBS -q batch

PBS -V

PBS -t 1-7

cd $PBS_O_WORKDIR FASTQFILE=$PBS_O_WORKDIR/inputfiles_hic_mouse_pac.txt; export FASTQFILE make --file /public/home/pengh/mytools//HiC-Pro_3.1.0/scripts/Makefile CONFIG_FILE=/hpdata/ws/mouse_hic/config-hicpro.txt CONFIG_SYS=/public/home/pengh/mytools//HiC-Pro_3.1.0/config-system.txt all_sub 2>&1

I tried running HiC-Pro with a total of 320 threads on ten nodes using Torque. According to the output pestat, all 10 nodes were indeed utilized, but only the first node is under high load. It seems that the remaining 9 nodes are not running any tasks.

YCMaCY commented 5 months ago

there are 20 nodes in my cluster

nservant commented 5 months ago

Hi, I think it mainly depends on your cluster ... on my side, I can only use one single node