Exception caught Cannot open file XXX/.bin_reads/paired_0.off': iostream error

ablab / spades

SPAdes Genome Assembler

http://ablab.github.io/spades/

Other

756 stars 137 forks source link

Exception caught Cannot open file XXX/.bin_reads/paired_0.off': iostream error #1250

Open gorfol opened 8 months ago

gorfol commented 8 months ago

Description of bug

Hi, Our server has 512 GB of RAM and 256 cores. I have an issue shown in the spades.log. If I run spades with root privileges, the results are the same. There is a paired_0.off file in the .bin_reads directory. We had an SSD change before this error happened, but the Linux and SPAdes version (however compiled again on the new system), the fastq files and parameters are the same. Thank you!

spades.log

params.txt

SPAdes version

3.15.5

Operating System

Ubuntu 22.04.1

Python Version

Python 3.11.5

Method of SPAdes installation

manual

No errors reported in spades.log

[X] Yes

asl commented 8 months ago

Hello

There is an I/O problem. SPAdes was unable to read one of its intermediate files. So, maybe more information is available in the system log? Does moving output / scratch dir to other volume helps?

gorfol commented 8 months ago

Thank you! Unfortunately I cannot find anything in the syslog and moving the whole spades directory to a mounted NAS drive does not solve the problem. I see the intermediate file and can access by my user, so I do not know why spades cannot handle it.

asl commented 8 months ago

Does it reproduce with another dataset? How about running spades.py --test?

gorfol commented 8 months ago

Yes, I tried different datasets.

The test ran with this result: == Running: /home/gorfol/miniconda3/bin/python /home/gorfol/spades-3.15.5/assembler/src/spades_pipeline/scripts/check_test_script.py --mode common --result_contigs_filename /home/gorfol/spades-3.15.5/assembler/spades_test/contigs.fasta --result_scaffolds_filename /home/gorfol/spades-3.15.5/assembler/spades_test/scaffolds.fasta

========= TEST PASSED CORRECTLY. . . . ======= SPAdes pipeline finished WITH WARNINGS!

=== Error correction and assembling warnings:

0:00:00.957 1M / 280M WARN General (launcher.cpp : 178) Your data seems to have high uniform coverage depth. It is strongly recommended to use --isolate option. ======= Warnings saved to /home/gorfol/spades-3.15.5/assembler/spades_test/warnings.log

gorfol commented 3 months ago

Without specifying the threads option, it runs smoothly, but of course it would be good if I could use all or most of the cores of the server...

asl commented 3 months ago

Without specifying the threads option, it runs smoothly, but of course it would be good if I could use all or most of the cores of the server...

You might probably check the open file limit (see ulimit -n). It might be set too low.