search

ngless-toolkit / ngless

NGLess: NGS with less work
https://ngless.embl.de
Other
142 stars 24 forks source link

Error: resource vanished (Broken pipe) #121

Closed ycl6 closed 4 years ago

ycl6 commented 5 years ago

Hi,

I seems if I had a job running IGC mapping, and while it is underway I start another job running IGC mapping, I will receive the broken pipe error.

[Wed 18-09-2019 10:58:47] Line 15: Starting mapping to /home/x/.local/share/ngless/data/Modules/igc.ngm/0.9/igc.fna
[Wed 18-09-2019 10:58:47] Line 15: Calling: /home/x/.local/share/ngless/bin/ngless-1.0.1-bwa mem -t 2 /home/x/.local/share/ngless/data/Modules/igc.ngm/0.9/igc-bwa-0.7.17.fna -p -
Exiting after internal error. If you can reproduce this issue, please run your script with the --trace flag and report a bug at https://github.com/ngless-toolkit/ngless/issues
fd:17: hClose: resource vanished (Broken pipe)
luispedro commented 5 years ago

I think I have seen this if bwa runs out of memory at just the wrong time for NGLess to catch it, so it freaks out incorrectly.

ycl6 commented 5 years ago

Hi @luispedro

Thanks for pointing that out. I checked the issue by running bwa and indeed found bwa complains running out of memory.

/home/x/.local/share/ngless/bin/ngless-1.0.1-bwa mem -t 2 /home/x/.local/share/ngless/data/Modules/igc.ngm/0.9/igc-bwa-0.7.17.fna test.1.fq.gz test.2.fq.gz > test.sam
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[bwa_idx_load_from_disk] Failed to allocate 1859039014 bytes at bwa.c line 281: Cannot allocate memory
luispedro commented 5 years ago

Obviously, this is still a bug in NGLess, which should give you a better error message.

mettejoe commented 1 year ago

I get the same error when I try to map a small fastq file against a very small fasta file. So in my case I don't think it is a memory issue as I have no problem mapping the same file against IGC. So I think it is the general error message when bwa fails for some reason. I will keep you updated.