search

epi2me-labs / wf-human-variation

Other
103 stars 44 forks source link

Process `bam_ingress:minimap2_alignment (1)` terminated with an error exit status (137) #133

Open Yoon90 opened 9 months ago

Yoon90 commented 9 months ago

Ask away!

Hello! nextflow.log

Human-variation-workflow has stopped with an error exit status(137) I guess it has occurred because of the shortage of RAM.

I'm attaching the head and the middle part of the nextflow.config

I found that the minimal required memory is 32GB but this computer cannot provide that much. Is there any option to change/lower required Memory(RAM)?

Logs:

[0;92m|||||||||| _ __ _ __ ____ |||||||||| | ____| _ | | \/ | __| | | | | ||||| | | | |) | | ) | |\/| | _| ___| |/ ` | ' \/ | ||||| | |_| /| | / /| | | | |_|| | (| | |) _ \ |||||||||| |____|_| |_|___|| ||| ||\,|._/|/ |||||||||| wf-human-variation v1.9.2 -------------------------------------------------------------------------------- Core Nextflow options runName : tender_pike containerEngine : docker container : ontresearch/wf-human-variation:sha0800eade05e4cbb75d45421633c78c4f6320b2f6 launchDir : /home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2 workDir : /home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2/work projectDir : /home/heeji/epi2melabs/workflows/epi2me-labs/wf-human-variation userName : heeji profile : standard configFiles : /home/heeji/epi2melabs/workflows/epi2me-labs/wf-human-variation/nextflow.config

Workflow Options sv : true snp : true cnv : true str : true

Main options sample_name : DBS_barcode05 bam : /media/heeji/2023_WES/nanopore/231227_DBS_4_pilot_test/no_sample/20231227_1908_X5_FAU99700_eaea91de/bam_pass/barcode05/merge_barcode05.bam ref : /media/heeji/2023_WES/hg38.fa bam_min_coverage: 5 out_dir : /home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2/output

!! Only displaying parameters that differ from the pipeline defaults !! -------------------------------------------------------------------------------- If you use epi2me-labs/wf-human-variation for your analysis please cite:

-------------------------------------------------------------------------------- This is epi2me-labs/wf-human-variation v1.9.2.

Caused by: Process bam_ingress:minimap2_alignment (1) terminated with an error exit status (137)

Command executed:

samtools bam2fq -@ 1 -T 1 merge_barcode05.bam | minimap2 -y -t 8 -ax map-ont hg38.fa - | samtools sort -@ 3 --write-index -o DBS_barcode05.bam##idx##DBS_barcode05.bam.bai -O bam --reference hg38.fa -

Command exit status: 137

Command output: (empty)

Command error: [M::mm_idx_gen::116.4521.80] collected minimizers [M::mm_idx_gen::217.7851.78] sorted minimizers [M::main::217.7921.78] loaded/built the index for 455 target sequence(s) [M::mm_mapopt_update::219.4621.78] mid_occ = 728 [M::mm_idx_stat] kmer size: 15; skip: 10; is_hpc: 0; #seq: 455 [M::mm_idx_stat::220.401*1.77] distinct minimizers: 100202295 (37.96% are singletons); average occurrences: 5.732; average spacing: 5.587; total length: 3209286105 [M::bam2fq_mainloop] discarded 0 singletons [M::bam2fq_mainloop] processed 236000 reads [bam_sort_core] merging from 0 files and 3 in-memory blocks... .command.sh: line 2: 25 Done samtools bam2fq -@ 1 -T 1 merge_barcode05.bam 26 Killed | minimap2 -y -t 8 -ax map-ont hg38.fa - 28 | samtools sort -@ 3 --write-index -o DBS_barcode05.bam##idx##DBS_barcode05.bam.bai -O bam --reference hg38.fa -

SamStudio8 commented 9 months ago

It looks like the alignment process ran out of memory, we are imminently releasing a new version of the workflow with more generous memory directives that should alleviate this.

On Mon, 22 Jan 2024, 09:03 Yoon, @.***> wrote:

Ask away!

Hello! nextflow.log https://github.com/epi2me-labs/wf-human-variation/files/14007577/nextflow.log

Human-variation-workflow has stopped with an error exit status(137) I guess it has occurred because of the shortage of RAM.

I'm attaching the head and the middle part of the nextflow.config

I found that the minimal required memory is 32GB but this computer cannot provide that much. Is there any option to change/lower required Memory(RAM)?

Logs:

[0;92m|||||||||| �[0m�[2m_ __ _ __ _ ____ �[0;92m|||||||||| �[0m�[2m|

| | | / | | | | | | _ �[0;33m||||| �[0m�[2m| | | |) | | _) | |/| | | | |/ ` | ' / | �[0;33m||||| �[0m�[2m| || /| | / /| | | | ||| | (| | |) �[0;94m|||||||||| �[0m�[2m||| |||| ||| ||_,|./|__/ �[0;94m|||||||||| �[0m�[1mwf-human-variation v1.9.2�[0m

�[2m--------------------------------------------------------------------------------�[0m �[1mCore Nextflow options�[0m �[0;34mrunName : �[0;32mtender_pike�[0m �[0;34mcontainerEngine : �[0;32mdocker�[0m �[0;34mcontainer : �[0;32montresearch/wf-human-variation:sha0800eade05e4cbb75d45421633c78c4f6320b2f6�[0m �[0;34mlaunchDir : �[0;32m/home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2�[0m �[0;34mworkDir : �[0;32m/home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2/work�[0m �[0;34mprojectDir : �[0;32m/home/heeji/epi2melabs/workflows/epi2me-labs/wf-human-variation�[0m �[0;34muserName : �[0;32mheeji�[0m �[0;34mprofile : �[0;32mstandard�[0m �[0;34mconfigFiles : �[0;32m/home/heeji/epi2melabs/workflows/epi2me-labs/wf-human-variation/nextflow.config�[0m

�[1mWorkflow Options�[0m �[0;34msv : �[0;32mtrue�[0m �[0;34msnp : �[0;32mtrue�[0m �[0;34mcnv : �[0;32mtrue�[0m �[0;34mstr : �[0;32mtrue�[0m

�[1mMain options�[0m �[0;34msample_name : �[0;32mDBS_barcode05�[0m �[0;34mbam : �[0;32m/media/heeji/2023_WES/nanopore/231227_DBS_4_pilot_test/no_sample/20231227_1908_X5_FAU99700_eaea91de/bam_pass/barcode05/merge_barcode05.bam�[0m �[0;34mref : �[0;32m/media/heeji/2023_WES/hg38.fa�[0m �[0;34mbam_min_coverage: �[0;32m5�[0m �[0;34mout_dir : �[0;32m/home/heeji/epi2melabs/instances/wf-human-variation_01HMQ5WY7W4NNY8QABXDCK57S2/output�[0m

!! Only displaying parameters that differ from the pipeline defaults !!

�[2m--------------------------------------------------------------------------------�[0m If you use epi2me-labs/wf-human-variation for your analysis please cite:

�[2m--------------------------------------------------------------------------------�[0m This is epi2me-labs/wf-human-variation v1.9.2.

Caused by: Process bam_ingress:minimap2_alignment (1) terminated with an error exit status (137)

Command executed:

samtools bam2fq -@ 1 -T 1 merge_barcode05.bam | minimap2 -y -t 8 -ax map-ont hg38.fa - | samtools sort -@ 3 --write-index -o DBS_barcode05.bam##idx##DBS_barcode05.bam.bai -O bam --reference hg38.fa -

Command exit status: 137

Command output: (empty)

Command error: [M::mm_idx_gen::116.452 1.80] collected minimizers [M::mm_idx_gen::217.7851.78] sorted minimizers [M::main::217.792 1.78] loaded/built the index for 455 target sequence(s) [M::mm_mapopt_update::219.4621.78] mid_occ = 728 [M::mm_idx_stat] kmer size: 15; skip: 10; is_hpc: 0; #seq: 455 [M::mm_idx_stat::220.401*1.77] distinct minimizers: 100202295 (37.96% are singletons); average occurrences: 5.732; average spacing: 5.587; total length: 3209286105 [M::bam2fq_mainloop] discarded 0 singletons [M::bam2fq_mainloop] processed 236000 reads [bam_sort_core] merging from 0 files and 3 in-memory blocks... .command.sh: line 2: 25 Done samtools bam2fq -@ 1 -T 1 merge_barcode05.bam 26 Killed | minimap2 -y -t 8 -ax map-ont hg38.fa - 28 | samtools sort -@ 3 --write-index -o DBS_barcode05.bam##idx##DBS_barcode05.bam.bai -O bam --reference hg38.fa -

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-human-variation/issues/133, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIN6OXDJUG6AWXWHTN3JZLYPYTNZAVCNFSM6AAAAABCE2PTPKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA4TGNBRGM3TMNI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

RenzoTale88 commented 9 months ago

@Yoon90 as @SamStudio8 mentioned, this is an out of memory issue, therefore decrease the memory usage won't fix the issue.

Yoon90 commented 9 months ago

@RenzoTale88 As far as I know the memory usage could be increased. https://labs.epi2me.io/trouble-shooting/#error-command-exit-status-137

But I guess this only works if the device has enough memory(more than 32GB). Should i wait for a new version?

cjw85 commented 9 months ago

You are correct, the workflow needs a minimum of 32Gb RAM to run, and very often the peak memory usage will be more than this. If your computer has only 32Gb RAM this workflow is unlikely to run reliably.