search

ablab / spades

SPAdes Genome Assembler
http://ablab.github.io/spades/
Other
763 stars 139 forks source link

Cannot open temporary file for writing. OS return value: 108 #1386

Closed ShyGuy509 closed 1 month ago

ShyGuy509 commented 1 month ago

Description of bug

Hello! I'm running spades (metaspades) on a cluster server to co-assemble 25 paired-end metagenomic samples. Based on the log it seemed to be running fine, and I've run the same job script on subsets of 5 of the same samples with no problem. However, during the K21 assembly it encountered the following error:

4:09:51.337 41G / 59G ERROR General (kmer_splitter.hpp : 147) Cannot open temporary file /lustre07/scratch/danross/metawrapJL/ALL/ASSEMBLY/tmp/spades_82ecu3os/construction_9bxNl9/kmers_raw_ky839K.24 for writing == Error == system call for: "['/cvmfs/soft.computecanada.ca/easybuild/software/2023/x86-64-v3/Compiler/gcccore/spades/3.15.4/bin/spades-core', '/lustre07/scratch/danross/metawrapJL/ALL/ASSEMBLY/K21/configs/config.info', '/lustre07/scratch/danross/metawrapJL/ALL/ASSEMBLY/K21/configs/mda_mode.info', '/lustre07/scratch/danross/metawrapJL/ALL/ASSEMBLY/K21/configs/meta_mode.info']" finished abnormally, OS return value: 108 None

It was using less than 2% of the available RAM and in a folder it should have every permission to write in. No other information is available. My samples are quite large, based on individual sample assembly I will need multiple TB of RAM - I've been waiting a month for my job to be accepted only to run into this error. Please help me figure out what happened. My command line code is below.

spades.py \ -1 ../CLEAN_READS/ALL_READS_1.fastq \ -2 ../CLEAN_READS/ALL_READS_2.fastq \ -m 4000 \ -t 64 \ --meta \ --only-assembler \ -o ../ASSEMBLY

spades.log

spades.log

params.txt

params.txt

SPAdes version

3.15.4

Operating System

Linux-4.18.0-553.16.1.el8_10.x86_64-x86_64-AMD_EPYC_7502_32-Core_Processor-with-glibc2.37

Python Version

3.11.4

Method of SPAdes installation

Unsure, on a third party cluster

No errors reported in spades.log

asl commented 1 month ago

The log reads:

  4:09:51.337    41G / 59G   ERROR   General                 (kmer_splitter.hpp         : 147)   Cannot open temporary file /lustre07/scratch/danross/metawrapJL/ALL/ASSEMBLY/tmp/spades_82ecu3os/construction_9bxNl9/kmers_raw_ky839K.24 for writing

This is an I/O problem on your cluster, not the SPAdes one. Maybe there is a limit on how many files you can open / write at once – oftentimes these clusters are very poorly configured. I would suggest you to contact your system administrator