search

ablab / spades

SPAdes Genome Assembler
http://ablab.github.io/spades/
Other
713 stars 131 forks source link

Test dataset does not work, spades-hammer fails #1272

Open Pigrenok opened 2 months ago

Pigrenok commented 2 months ago

Description of bug

I have an error on Spades 3.15.5 downloaded and installed as Binary from here (GitHub Releases). It is the same issue as reported in #1267. Unfortunately, it is marked as "outdated" and "wsl", but there is a second comment on the latest version and not in wsl with the same error. Hence, creating the new issue as tags are confusing.

It is run in Docker based on Ubuntu 22.04 image (run under normal user, not root):

$ uname -a
Linux 28c19eb34aa0 5.15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Python 3.10.12

spades.log

Command line: /tools/SPAdes-3.15.5-Linux/bin/spades.py --test

System information: SPAdes version: 3.15.5 Python version: 3.10.12 OS: Linux-5.15.0-102-generic-x86_64-with-glibc2.35

Output dir: /home/user1/2024-04-12-assembly/spades_test Mode: read error correction and assembling Debug mode is turned OFF

Dataset parameters: Standard mode For multi-cell/isolate data we recommend to use '--isolate' option; for single-cell MDA data use '--sc'; for metagenomic data use '--meta'; for RNA-Seq use '--rna'. Reads: Library number: 1, library type: paired-end orientation: fr left reads: ['/tools/SPAdes-3.15.5-Linux/share/spades/test_dataset/ecoli_1K_1.fq.gz'] right reads: ['/tools/SPAdes-3.15.5-Linux/share/spades/test_dataset/ecoli_1K_2.fq.gz'] interlaced reads: not specified single reads: not specified merged reads: not specified Read error correction parameters: Iterations: 1 PHRED offset will be auto-detected Corrected reads will be compressed Assembly parameters: k: automatic selection based on read length Repeat resolution is enabled Mismatch careful mode is turned OFF MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF Other parameters: Dir for temp files: /home/user1/2024-04-12-assembly/spades_test/tmp Threads: 16 Memory limit (in Gb): 62

======= SPAdes pipeline started. Log can be found here: /home/user1/2024-04-12-assembly/spades_test/spades.log

/tools/SPAdes-3.15.5-Linux/share/spades/test_dataset/ecoli_1K_1.fq.gz: max reads length: 100 /tools/SPAdes-3.15.5-Linux/share/spades/test_dataset/ecoli_1K_2.fq.gz: max reads length: 100

Reads length: 100

===== Before start started.

===== Read error correction started.

===== Read error correction started.

== Running: /tools/SPAdes-3.15.5-Linux/bin/spades-hammer /home/user1/2024-04-12-assembly/spades_test/corrected/configs/config.info

0:00:00.000 1M / 18M INFO General (main.cpp : 75) Starting BayesHammer, built from refs/heads/spades_3.15.5, git revision e757b8216f9a038fb616e9551a2d4891b2d19ad7 0:00:00.003 1M / 18M INFO General (main.cpp : 76) Loading config from /home/user1/2024-04-12-assembly/spades_test/corrected/configs/config.info 0:00:00.008 1M / 18M INFO General (main.cpp : 78) Maximum # of threads to use (adjusted due to OMP capabilities): 16 0:00:00.010 1M / 18M INFO General (memory_limit.cpp : 54) Memory limit set to 62 Gb 0:00:00.010 1M / 18M INFO General (main.cpp : 86) Trying to determine PHRED offset 0:00:00.011 1M / 18M INFO General (main.cpp : 92) Determined value is 33 0:00:00.011 1M / 18M INFO General (hammer_tools.cpp : 38) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ] 0:00:00.011 1M / 18M INFO General (main.cpp : 113) Size of aux. kmer data 24 bytes === ITERATION 0 begins === 0:00:00.011 1M / 18M INFO General (kmer_index_builder.hpp : 243) Splitting kmer instances into 16 files using 16 threads. This might take a while. The program was terminated by segmentation fault === Stack Trace === /tools/SPAdes-3.15.5-Linux/bin/spades-hammer() [0x421e29] /tools/SPAdes-3.15.5-Linux/bin/spades-hammer() [0x423956] /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f3a4127f520] /tools/SPAdes-3.15.5-Linux/bin/spades-hammer() [0x4847d4] /tools/SPAdes-3.15.5-Linux/bin/spades-hammer(cfree+0x85) [0x484a15] /tools/SPAdes-3.15.5-Linux/bin/spades-hammer() [0x494ffe] /tools/SPAdes-3.15.5-Linux/bin/spades-hammer() [0x495c65] /tools/SPAdes-3.15.5-Linux/bin/spades-hammer() [0x460f67] /tools/SPAdes-3.15.5-Linux/bin/spades-hammer() [0x450515] /tools/SPAdes-3.15.5-Linux/bin/spades-hammer() [0x461d47] /tools/SPAdes-3.15.5-Linux/bin/spades-hammer() [0x457100] /tools/SPAdes-3.15.5-Linux/bin/spades-hammer() [0x416300] /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f3a41266d90] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f3a41266e40] /tools/SPAdes-3.15.5-Linux/bin/spades-hammer() [0x420681]

== Error == system call for: "['/tools/SPAdes-3.15.5-Linux/bin/spades-hammer', '/home/user1/2024-04-12-assembly/spades_test/corrected/configs/config.info']" finished abnormally, OS return value: -11

In case you have troubles running SPAdes, you can write to spades.support@cab.spbu.ru or report an issue on our GitHub repository github.com/ablab/spades Please provide us with params.txt and spades.log files from the output directory.

SPAdes log can be found here: /home/user1/2024-04-12-assembly/spades_test/spades.log

Thank you for using SPAdes!

params.txt

Command line: /tools/SPAdes-3.15.5-Linux/bin/spades.py --test

System information: SPAdes version: 3.15.5 Python version: 3.10.12 OS: Linux-5.15.0-102-generic-x86_64-with-glibc2.35

Output dir: /home/user1/2024-04-12-assembly/spades_test Mode: read error correction and assembling Debug mode is turned OFF

Dataset parameters: Standard mode For multi-cell/isolate data we recommend to use '--isolate' option; for single-cell MDA data use '--sc'; for metagenomic data use '--meta'; for RNA-Seq use '--rna'. Reads: Library number: 1, library type: paired-end orientation: fr left reads: ['/tools/SPAdes-3.15.5-Linux/share/spades/test_dataset/ecoli_1K_1.fq.gz'] right reads: ['/tools/SPAdes-3.15.5-Linux/share/spades/test_dataset/ecoli_1K_2.fq.gz'] interlaced reads: not specified single reads: not specified merged reads: not specified Read error correction parameters: Iterations: 1 PHRED offset will be auto-detected Corrected reads will be compressed Assembly parameters: k: automatic selection based on read length Repeat resolution is enabled Mismatch careful mode is turned OFF MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF Other parameters: Dir for temp files: /home/user1/2024-04-12-assembly/spades_test/tmp Threads: 16 Memory limit (in Gb): 62

SPAdes version

v3.15.5

Operating System

Ubuntu 22.04

Python Version

3.10.12

Method of SPAdes installation

Binaries from GitHub Releases

No errors reported in spades.log

Pigrenok commented 2 months ago

Just an update. I changed the Spades version to 3.15.3 and now it is python error from PyYaml module (known issue in PyYaml v3).

Traceback (most recent call last):
  File "/tools/SPAdes-3.15.3-Linux/bin/spades.py", line 659, in <module>
    main(sys.argv)
  File "/tools/SPAdes-3.15.3-Linux/bin/spades.py", line 599, in main
    print_params(log, log_filename, command_line, args, cfg)
  File "/tools/SPAdes-3.15.3-Linux/bin/spades.py", line 327, in print_params
    print_used_values(cfg, log)
  File "/tools/SPAdes-3.15.3-Linux/bin/spades.py", line 117, in print_used_values
    dataset_data = pyyaml.load(open(cfg["dataset"].yaml_filename))
  File "/tools/SPAdes-3.15.3-Linux/share/spades/pyyaml3/__init__.py", line 72, in load
    return loader.get_single_data()
  File "/tools/SPAdes-3.15.3-Linux/share/spades/pyyaml3/constructor.py", line 37, in get_single_data
    return self.construct_document(node)
  File "/tools/SPAdes-3.15.3-Linux/share/spades/pyyaml3/constructor.py", line 46, in construct_document
    for dummy in generator:
  File "/tools/SPAdes-3.15.3-Linux/share/spades/pyyaml3/constructor.py", line 398, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/tools/SPAdes-3.15.3-Linux/share/spades/pyyaml3/constructor.py", line 204, in construct_mapping
    return super().construct_mapping(node, deep=deep)
  File "/tools/SPAdes-3.15.3-Linux/share/spades/pyyaml3/constructor.py", line 126, in construct_mapping
    if not isinstance(key, collections.Hashable):
AttributeError: module 'collections' has no attribute 'Hashable'

It seems Spades uses internal PyYaml instead of the one available in system python.

Which version would be working stable in Python 3.10?

Pigrenok commented 2 months ago

OK. I tried to build 3.15.5 and after building it seems to be working fine. You probably would like to state which system the pre-built binaries are compiled for and which ones are supported.