Aligning with HiC-Pro Singularity Image and parallel mode

[joreynajr]

I am trying to run HiC-Pro using the Singularity Image and try to use the -p argument. Below I am posting the version, my snakemake/bash code and the error I get: Version: 2.11.4 Code:

    # getting absolute paths for data and outdirs, required
    # by HiCPro
    abs_datadir=$(readlink -f {params.datadir})
    abs_outdir=$(readlink -f {params.outdir})

    # running with setting -s so that it runs only the alignment
    # pipeline (default settings)
    HiC-Pro -s mapping \
            -p \
            -i $abs_datadir \
            -o $abs_outdir \
            -c {input.config} >> {log} 2>&1

    # running the qsub script generated by HiC-Pro
    qids=$(qsub {params.qscript}) >> {log} 2>&1
    mkdir -p $(basename {output.bowtie_running}) >> {log} 2>&1
    echo $qids > {output.bowtie_running}

Error:

    make: /usr/local/bin/HiC-Pro_2.11.4/scripts/Makefile: No such file or directory
    make: *** No rule to make target '/usr/local/bin/HiC-Pro_2.11.4/scripts/Makefile'.  Stop.

My code is basically generating the HiCPro_step1_.sh script and then I'm qsubbing that script with qids=$(qsub {params.qscript}) >> {log} 2>&1 but all of this should be pretty standard, what the error suggests to me is that there is some disconnect between the Singularity Image and the qsub script file? I'm pasting HiCPro_step1_.sh just to show that it as generated and maybe it can help us figure out how to connect it back to the qsub script.

#!/bin/bash
#PBS -l nodes=1:ppn=4,mem=64000M,walltime=200:00:00
#PBS -M jreyna@lji.org
#PBS -m ae
#PBS -j eo
#PBS -N HiCpro_s1_
#PBS -q default
#PBS -V
#PBS -t 1-2
cd $PBS_O_WORKDIR
FASTQFILE=$PBS_O_WORKDIR/inputfiles_.txt; export FASTQFILE
make --file Makefile CONFIG_FILE=/home/jreyna/jreyna/projects/HiCnv/results/refs/hicpro/config-hicpro.hindiii.txt CONFIG_SYS=/usr/local/bin/HiC-Pro_2.11.4/config-system.txt mapping 2>&1

Thanks for the help!

[joreynajr]

I did a little digging and the problem seems to be in HiCPro_step1_.sh. The script does cd $PBS_O_WORKDIR but the Makefile is not there. I guess it can also be pathed absolutely? At least for Singularity image users. I tried changing the HiCPro_step1_.sh into:

 
#!/bin/bash
#PBS -l nodes=1:ppn=4,mem=64000M,walltime=200:00:00
#PBS -M jreyna@lji.org
#PBS -m ae
#PBS -j eo
#PBS -N HiCpro_s1_
#PBS -q default
#PBS -V
#PBS -t 1-2
cd $PBS_O_WORKDIR
FASTQFILE=$PBS_O_WORKDIR/inputfiles_.txt; export FASTQFILE
make --file /usr/local/bin/HiC-Pro_2.11.4/scripts/Makefile CONFIG_FILE=/home/jreyna/jreyna/projects/HiCnv/results/refs/hicpro/config-hicpro.hindiii.txt CONFIG_SYS=/usr/local/bin/HiC-Pro_2.11.4/config-system.txt mapping 2>&1

The above works but I had to run the command from inside of Singularity so if qsub is going to work it seems we would have to do something like:

 
#!/bin/bash
#PBS -l nodes=1:ppn=4,mem=64000M,walltime=200:00:00
#PBS -M jreyna@lji.org
#PBS -m ae
#PBS -j eo
#PBS -N HiCpro_s1_
#PBS -q default
#PBS -V
#PBS -t 1-2

cmd="cd $PBS_O_WORKDIR;"
cmd+="FASTQFILE=$PBS_O_WORKDIR/inputfiles_.txt; export FASTQFILE;"
cmd+="make --file /usr/local/bin/HiC-Pro_2.11.4/scripts/Makefile CONFIG_FILE=/home/jreyna/jreyna/projects/HiCnv/results/refs/hicpro/config-hicpro.hindiii.txt CONFIG_SYS=/usr/local/bin/HiC-Pro_2.11.4/config-system.txt mapping 2>&1;"
singularity exec $cmd

Do you think it should be run this way?

[nservant]

Hi, Indeed, the qusb script is not set up to run with Singularity. Running singularity exec should do the job. Does it work ? N

[joreynajr]

I ended up installing the Anaconda environment (from Using HiC-Pro through conda) and installing HiC-Pro with make (from How to install it ?). I just kept running into problems trying to change HiCProstep1.sh and trying to run singularity exec from inside of it. After installing the full HiC-Pro version I can now run HiC-Pro in parallel mode.

Thanks for the help!

Joaquin