search

NationalGenomicsInfrastructure / piper

A genomics pipeline build on top of the GATK Queue framework
9 stars 9 forks source link

UPPMAX: Memory overflow error #36

Closed parlundin closed 9 years ago

parlundin commented 9 years ago

SLURM job killed by an out-of-memory condition. Running SBATCH with -p node and -n 16

Seems to be a combination of bwa and Samtools that uses to much ram.

/pica/v9/a2014205_nobackup/perlu/ANALYSIS/ND-0522/.queue/tmp/.exec8811658286087080852: line 2:  6923 Broken pipe             /sw/apps/bioinfo/bwa/0.7.5a/milou/bin/bwa mem -M -t 15 -R "@RG\tID:AC45JGANXX.NA11993_Nano.7\\tSM:NA11993_Nano\\tCN:NGI\\tLB:SX398_NA11993_Nano\\tPL:Illumina\\tPU:AC45JGANXX.NA11993_Nano.7" /proj/a2014205/piper_references/gatk_bundle/2.8/b37/human_g1k_v37.fasta /pica/v9/a2014205_nobackup/perlu/Subsampled2/DATA/ND-0522/NA11993_Nano/SX398_NA11993_Nano/140821_D00458_0029_AC45JGANXX/NA11993_Nano_TAGCTT_L007_R1_001.fastq.gz /pica/v9/a2014205_nobackup/perlu/Subsampled2/DATA/ND-0522/NA11993_Nano/SX398_NA11993_Nano/140821_D00458_0029_AC45JGANXX/NA11993_Nano_TAGCTT_L007_R2_001.fastq.gz
      6924                       | /sw/apps/bioinfo/samtools/0.1.19/milou/bin/samtools view -Su -
      6925 Killed                  | /sw/apps/bioinfo/samtools/0.1.19/milou/bin/samtools sort -@ 15 -m 7G - /pica/v9/a2014205_nobackup/perlu/ANALYSIS/ND-0522/pipeline_output/01_raw_alignments/NA11993_Nano.AC45JGANXX.NA11993_Nano.7
mariogiov commented 9 years ago

@francesco didn't you encounter something like this once? I recall you manually setting the Java memory parameters somehow,

vezzi commented 9 years ago

I do not think the things are connected.... this seems the node to go out of memory my problem was running too many piper jobs on the login node

johandahlberg commented 9 years ago

I've been talking to @parlundin about this, and it's bwa + samtools that cause the problem. I'll fix it for the next release.

johandahlberg commented 9 years ago

I think that this should be ok now. So I'll close this.