search

ablab / spades

SPAdes Genome Assembler
http://ablab.github.io/spades/
Other
739 stars 134 forks source link

getting error while running spades.py error : finished abnormally, OS return value: -11 #1248

Open rishidash12 opened 7 months ago

rishidash12 commented 7 months ago

Description of bug

I am running the following command "spades.py --rna -s mouse1_mRNA.fastq -o mouse1_spades -m 50 --phred-offset 33" getting error like this == Error == system call for: "['/home/ccmr_server/metatranscriptomics/SPAdes-3.15.5-Linux/bin/spades-core', '/home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/config.info', '/home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/mda_mode.info', '/home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/rna_mode.info']" finished abnormally, OS return value: -11

initially ram and phred-offset i kept default then i reduced ram to 100 then 50 still got the same error. Please kindly help me to solve this.

spades.log

Command line: /home/ccmr_server/metatranscriptomics/SPAdes-3.15.5-Linux/bin/spades.py --rna -s /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_mRNA.fastq -o /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades -m 50 --phred-offset 33

System information: SPAdes version: 3.15.5 Python version: 3.10.12 OS: Linux-5.15.0-91-generic-x86_64-with-glibc2.35

Output dir: /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades Debug mode is turned OFF

Dataset parameters: RNA-seq mode Reads: Library number: 1, library type: single left reads: not specified right reads: not specified interlaced reads: not specified single reads: ['/home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_mRNA.fastq'] merged reads: not specified Assembly parameters: k: automatic selection based on read length Repeat resolution is enabled Mismatch careful mode is turned OFF MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF Other parameters: Dir for temp files: /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/tmp Threads: 16 Memory limit (in Gb): 50

======= SPAdes pipeline started. Log can be found here: /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/spades.log

/home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_mRNA.fastq: max reads length: 160

Reads length: 160

K values to be used: [53, 79]

===== Before start started.

===== Assembling started.

===== K53 started.

== Running: /home/ccmr_server/metatranscriptomics/SPAdes-3.15.5-Linux/bin/spades-core /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/config.info /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/mda_mode.info /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/rna_mode.info

0:00:00.000 1M / 17M INFO General (main.cpp : 99) Loaded config from /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/config.info 0:00:00.000 1M / 17M INFO General (main.cpp : 99) Loaded config from /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/mda_mode.info 0:00:00.000 1M / 17M INFO General (main.cpp : 99) Loaded config from /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/rna_mode.info 0:00:00.000 1M / 17M INFO General (memory_limit.cpp : 54) Memory limit set to 50 Gb 0:00:00.000 1M / 17M INFO General (main.cpp : 107) Starting SPAdes, built from refs/heads/spades_3.15.5, git revision e757b8216f9a038fb616e9551a2d4891b2d19ad7 0:00:00.000 1M / 17M INFO General (main.cpp : 108) Maximum k-mer length: 128 0:00:00.000 1M / 17M INFO General (main.cpp : 109) Assembling dataset (/home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/dataset.info) with K=53 0:00:00.000 1M / 17M INFO General (main.cpp : 110) Maximum # of threads to use (adjusted due to OMP capabilities): 16 0:00:00.000 1M / 17M INFO General (pipeline.cpp : 212) SPAdes started 0:00:00.000 1M / 17M INFO General (pipeline.cpp : 225) Starting from stage: read_conversion 0:00:00.000 1M / 17M INFO General (pipeline.cpp : 231) Two-step repeat resolution disabled 0:00:00.000 1M / 17M INFO GraphCore (graph_core.hpp : 672) Graph created, vertex min_id: 3, edge min_id: 3 0:00:00.000 1M / 17M INFO GraphCore (graph_core.hpp : 673) Vertex size: 48, edge size: 40 0:00:00.001 1M / 17M INFO General (edge_index.hpp : 113) Size of edge index entries: 12/8 0:00:00.001 1M / 17M INFO StageManager (stage.cpp : 185) STAGE == Binary Read Conversion (id: read_conversion) 0:00:00.001 1M / 17M INFO General (read_converter.cpp : 72) Converting reads to binary format for library #0 (takes a while) 0:00:00.001 1M / 17M INFO General (read_converter.cpp : 73) Converting paired reads 0:00:00.001 8M / 17M INFO General (binary_converter.cpp : 111) 0 reads written 0:00:00.001 1M / 17M INFO General (read_converter.cpp : 86) Converting single reads 0:00:00.129 11M / 24M INFO General (binary_converter.cpp : 96) 16384 reads processed 0:00:00.134 11M / 24M INFO General (binary_converter.cpp : 96) 32768 reads processed 0:00:00.145 11M / 25M INFO General (binary_converter.cpp : 96) 65536 reads processed 0:00:00.304 11M / 26M INFO General (binary_converter.cpp : 111) 82925 reads written 0:00:00.304 7M / 26M INFO General (read_converter.cpp : 92) Converting merged reads 0:00:00.305 11M / 26M INFO General (binary_converter.cpp : 111) 0 reads written 0:00:00.307 1M / 26M INFO StageManager (stage.cpp : 185) STAGE == de Bruijn graph construction (id: construction) 0:00:00.310 1M / 26M INFO General (construction.cpp : 153) Max read length 160 0:00:00.310 1M / 26M INFO General (construction.cpp : 159) Average read length 137.482 0:00:00.310 1M / 26M INFO General (stage.cpp : 117) PROCEDURE == k+1-mer counting (id: construction:kpomer_counting) 0:00:00.310 1M / 26M INFO General (kmer_index_builder.hpp : 243) Splitting kmer instances into 160 files using 16 threads. This might take a while.

== Error == system call for: "['/home/ccmr_server/metatranscriptomics/SPAdes-3.15.5-Linux/bin/spades-core', '/home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/config.info', '/home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/mda_mode.info', '/home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/K53/configs/rna_mode.info']" finished abnormally, OS return value: -11

In case you have troubles running SPAdes, you can write to spades.support@cab.spbu.ru or report an issue on our GitHub repository github.com/ablab/spades Please provide us with params.txt and spades.log files from the output directory.

SPAdes log can be found here: /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/spades.log

Thank you for using SPAdes!

params.txt

Command line: /home/ccmr_server/metatranscriptomics/SPAdes-3.15.5-Linux/bin/spades.py --rna -s /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_mRNA.fastq -o /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades -m 50 --phred-offset 33

System information: SPAdes version: 3.15.5 Python version: 3.10.12 OS: Linux-5.15.0-91-generic-x86_64-with-glibc2.35

Output dir: /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades Debug mode is turned OFF

Dataset parameters: RNA-seq mode Reads: Library number: 1, library type: single left reads: not specified right reads: not specified interlaced reads: not specified single reads: ['/home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_mRNA.fastq'] merged reads: not specified Assembly parameters: k: automatic selection based on read length Repeat resolution is enabled Mismatch careful mode is turned OFF MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF Other parameters: Dir for temp files: /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades/tmp Threads: 16 Memory limit (in Gb): 50

SPAdes version

SPAdes v3.15.5

Operating System

Linux-5.15.0-91-generic-x86_64-with-glibc2.35

Python Version

Python v3.10.12

Method of SPAdes installation

binaries

No errors reported in spades.log

asl commented 7 months ago

You might hit limit of # of opened files. Likely more information could be found in the system log.

rishidash12 commented 7 months ago

I find system log files don't know which one to see for the spades errors.

Screenshot from 2024-02-22 10-04-37

kovcsboti commented 7 months ago

I got the same error message when I ran spades 3.15.4 on bacterial genome, however it ran fine with spades 3.13.1 and it ran without problem (not sure if it helps, just letting you know).

rishidash12 commented 7 months ago

@kovcsboti I tried using 3.13.1 version of SPAdes but now its showing a different error. Not hashable. /home/ccmr_server/metatranscriptomics/new_workshop_file/SPAdes-3.13.1-Linux/bin/spades.py:13: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives from distutils import dir_util Command line: /home/ccmr_server/metatranscriptomics/new_workshop_file/SPAdes-3.13.1-Linux/bin/spades.py --rna -s /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_mRNA.fastq -o /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades

System information: SPAdes version: 3.13.1 Python version: 3.10.12 OS: Linux-5.15.0-91-generic-x86_64-with-glibc2.35

Output dir: /home/ccmr_server/metatranscriptomics/new_workshop_file/mouse1_spades Mode: ONLY assembling (without read error correction) Debug mode is turned OFF

Dataset parameters: RNA-seq mode Reads: Traceback (most recent call last): File "/home/ccmr_server/metatranscriptomics/new_workshop_file/SPAdes-3.13.1-Linux/bin/spades.py", line 1102, in main(sys.argv) File "/home/ccmr_server/metatranscriptomics/new_workshop_file/SPAdes-3.13.1-Linux/bin/spades.py", line 721, in main print_used_values(cfg, log) File "/home/ccmr_server/metatranscriptomics/new_workshop_file/SPAdes-3.13.1-Linux/bin/spades.py", line 110, in print_used_values dataset_data = pyyaml.load(open(cfg["dataset"].yaml_filename, 'r')) File "/home/ccmr_server/metatranscriptomics/new_workshop_file/SPAdes-3.13.1-Linux/share/spades/pyyaml3/init.py", line 72, in load return loader.get_single_data() File "/home/ccmr_server/metatranscriptomics/new_workshop_file/SPAdes-3.13.1-Linux/share/spades/pyyaml3/constructor.py", line 37, in get_single_data return self.construct_document(node) File "/home/ccmr_server/metatranscriptomics/new_workshop_file/SPAdes-3.13.1-Linux/share/spades/pyyaml3/constructor.py", line 46, in construct_document for dummy in generator: File "/home/ccmr_server/metatranscriptomics/new_workshop_file/SPAdes-3.13.1-Linux/share/spades/pyyaml3/constructor.py", line 398, in construct_yaml_map value = self.construct_mapping(node) File "/home/ccmr_server/metatranscriptomics/new_workshop_file/SPAdes-3.13.1-Linux/share/spades/pyyaml3/constructor.py", line 204, in construct_mapping return super().construct_mapping(node, deep=deep) File "/home/ccmr_server/metatranscriptomics/new_workshop_file/SPAdes-3.13.1-Linux/share/spades/pyyaml3/constructor.py", line 126, in construct_mapping if not isinstance(key, collections.Hashable): AttributeError: module 'collections' has no attribute 'Hashable'

asl commented 7 months ago

AttributeError: module 'collections' has no attribute 'Hashable'

This is a known issue fixed in 3.15.3. 3.13.1 is simply ancient and contains known issues

rishidash12 commented 7 months ago

But Sir @asl I am also facing issues in 3.15.5 version of Spades and i.e. finished abnormally, OS return value: -11. Please help me to resolve this problem.