Bowtie2 mem error？ qsub , SGE

[Ruismart]

I'm working with an old version (Red hat 5.7) system computer cluster server without Internet-connection. It took me a lot of time to install this atac_dnase_pipeline. Anyway, the software invironment seems available now. I have tried to submit analysis job using qsub. It always terminated like this:

Specified adapter for rep1:00 (PE) : None
Task has finished (23 seconds).
Task failed:    
        Program & line     : '***/modules/align_bowtie2.bds', line 148
        Task Name          : 'bowtie2_PE rep1'        
        Task hint          : 'bowtie2    -X2000 --mm --local --threads 6 -x '
        Task resources     : 'cpus: 6   mem: -1.0 B     wall-timeout: 8640000'
        State              : 'ERROR'
        Dependency state   : 'ERROR'
        Retries available  : '1'

This is my script.

#!/bin/bash
#$ -S /bin/bash
#$ -q all.q
#$ -j yes
#$ -cwd
#$ -N ***
#$ -o ***
#$ -l h_vmem=15g
#$ -l ht=blade
source ***/.bashrc
source ***/.bash_profile
bds ***/atac.bds \
-species hg38 \
-nth 6 \
-mem_bwt2 12g \
-out_dir *** \
-fastq1_1 ***R1.fastq.gz \
-fastq1_2 ***R2.fastq.gz

I have tried : bds -s sge -q all.q atac.bds ... commad，just the same error. And I'm sure someone has succeed using qsub.

So, what's the cause of this , the cluster didn't provide enough memory for the pipeline?

Thanks.

[leepc12]

Please post a full log for more info. Can you give more memory #$ -l h_vmem=30g and try again? For the first command (local mode using qsub), -mem_bwt2 12g is useless so remove it.

BTW Your second command (sge mode, in this mode pipeline automatically qsub/qstat each sub-task) is wrong (bds -s sge -q all.q atac.bds ...). You need to run bds -s sge atac.bds -q all.q ... or bds atac.bds -system sge -q all.q ....

[Ruismart]

Hi, I installed a standalone bowtie2 and tried this script with qsub :

#!/bin/bash
#$ -S /bin/bash
#$ -q all.q
#$ -j yes
#$ -cwd
#$ -N ***
#$ -o ***
#$ -l h_vmem=15g
#$ -l ht=blade
~/bowtie2 -X2000 --mm --local --threads 6 
-x ***/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta \
-1 ***R1.fastq.gz \
-2 ***R2.fastq.gz

It worked well. Then I realized this might not be a memory issue. I tested commands after source activate bds_atac , found out bowtie2/perl couldn't run like this:

***/miniconda3/envs/bds_atac/bin
$ ./perl --version
./perl: /lib64/libc.so.6: version `GLIBC_2.11' not found (required by ./perl)

It seemed that the old version(red hat 5.7 , gcc4.1.2, glibc 2.5) system env wasn't available completely. Then I linked it with a newly built gcc-4.9.2/glibc-2.19 lib:

***/patchelf --set-rpath \
***/glibc-2.19/lib:***/gcc-4.9.2/lib64:***gcc-4.9.2/lib:/usr/lib64:/usr/lib:/lib64:/lib:***/miniconda3/envs/bds_atac/lib \
--set-interpreter ***/glibc-2.19/lib/ld-2.19.so \
***/miniconda3/envs/bds_atac/bin/perl

My mentor had taught me that way to deal with a python lib compatible error. It also worked on perl . Then my previous bds script worked using qsub.

It's painful to have to use an old system. Hope it helps with other issues.