search

nservant / HiC-Pro

HiC-Pro: An optimized and flexible pipeline for Hi-C data processing
Other
386 stars 182 forks source link

Can multiple nodes be set for parallel processing in HiC-Pro (Torque)? #616

Closed YCMaCY closed 10 months ago

YCMaCY commented 10 months ago

cat HiCPro_step1_hic_mouse.sh

!/bin/bash

PBS -l nodes=10:ppn=32,mem=60gb,walltime=

PBS -M

PBS -m ae

PBS -j eo

PBS -N HiCpro_hic_mouse_pac

PBS -q batch

PBS -V

PBS -t 1-7

cd $PBS_O_WORKDIR FASTQFILE=$PBS_O_WORKDIR/inputfiles_hic_mouse_pac.txt; export FASTQFILE make --file /public/home/pengh/mytools//HiC-Pro_3.1.0/scripts/Makefile CONFIG_FILE=/hpdata/ws/mouse_hic/config-hicpro.txt CONFIG_SYS=/public/home/pengh/mytools//HiC-Pro_3.1.0/config-system.txt all_sub 2>&1

I tried running HiC-Pro with a total of 320 threads on ten nodes using Torque. According to the output pestat, all 10 nodes were indeed utilized, but only the first node is under high load. It seems that the remaining 9 nodes are not running any tasks.

YCMaCY commented 10 months ago

there are 20 nodes in my cluster

nservant commented 10 months ago

Hi, I think it mainly depends on your cluster ... on my side, I can only use one single node