search

ncbi / fcs

Foreign Contamination Screening caller scripts and documentation
Other
88 stars 12 forks source link

segment.cpp:506 in sort_and_merge(...): Assertion failed: (prev->pos < 0) == (ivl.pos < 0) #70

Closed eeaunin closed 4 months ago

eeaunin commented 4 months ago

Hello, Another error that causes crashes in my FCS-GX runs is with the message: Fatal error (St13runtime_error): segment.cpp:506 in sort_and_merge(...): Assertion failed: (prev->pos < 0) == (ivl.pos < 0)

Here is a longer piece of log of an FCS-GX run that crashed with this error:

=============================================================================== 
Source:      /mft-volume 
Destination: /app/db/gxdb 
Resuming failed transfer in /app/db/gxdb... 
Space check: Available:1.27TiB; Existing:0B; Incoming:464.34GiB; Delta:464.34GiB

Requires transfer: 59B all.meta.jsonl 
Copying /mft-volume/all.meta.jsonl to /app/db/gxdb/all.meta.jsonl.part... 

Requires transfer: 187B all.README.txt 
Copying /mft-volume/all.README.txt to /app/db/gxdb/all.README.txt.part... 

Requires transfer: 6.09MiB all.taxa.tsv 
Copying /mft-volume/all.taxa.tsv to /app/db/gxdb/all.taxa.tsv.part... 

Requires transfer: 7.86MiB all.blast_div.tsv.gz 
Copying /mft-volume/all.blast_div.tsv.gz to /app/db/gxdb/all.blast_div.tsv.gz.part... 

Requires transfer: 8.48MiB all.assemblies.tsv 
Copying /mft-volume/all.assemblies.tsv to /app/db/gxdb/all.assemblies.tsv.part... 

Requires transfer: 21.51MiB all.seq_info.tsv.gz 
Copying /mft-volume/all.seq_info.tsv.gz to /app/db/gxdb/all.seq_info.tsv.gz.part... 

Requires transfer: 165.14GiB all.gxs 
Copying /mft-volume/all.gxs to /app/db/gxdb/all.gxs.part... 

Requires transfer: 299.16GiB all.gxi 
Copying /mft-volume/all.gxi to /app/db/gxdb/all.gxi.part... 
Done. 

--------------------------------------------------------------------

tax-id    : 339868
fasta     : /sample-volume/assembly.fasta
size      : 23811.64 MiB
split-fa  : True
BLAST-div : salamanders
gx-div    : anml:amphibians
w/same-tax: True
bin-dir   : /app/bin
gx-db     : /app/db/gxdb/jt8_gx_mapper_16995/all.gxi
gx-ver    : Mar 10 2023 15:34:33; git:v0.4.0-3-g8096f62
output    : /output-volume//assembly.339868.taxonomy.rpt

--------------------------------------------------------------------

Using GX_PREFETCH=0
Collecting masking statistics...
Fatal error (St13runtime_error): segment.cpp:506 in sort_and_merge(...): Assertion failed: (prev->pos < 0) == (ivl.pos < 0)
Error: Process failed with retcode -13: ['gzip', '-cdf', '/sample-volume/assembly.fasta'])
Error: Process failed with retcode 1: ['/app/bin/gx', 'split-fasta'])
Traceback (most recent call last):
  File "/tmp/Bazel.runfiles_ab94s9vi/runfiles/cgr_fcs/apps/private/run_gx/run_gx.py", line 864, in <module>
    main()
  File "/tmp/Bazel.runfiles_ab94s9vi/runfiles/cgr_fcs/apps/private/run_gx/run_gx.py", line 844, in main
    run_gx_pipeline(args)
  File "/tmp/Bazel.runfiles_ab94s9vi/runfiles/cgr_fcs/apps/private/run_gx/run_gx.py", line 539, in run_gx_pipeline
    run(p_zcat_fasta, p_save_hits, p_main)
  File "/tmp/Bazel.runfiles_ab94s9vi/runfiles/cgr_fcs/apps/private/run_gx/run_gx.py", line 209, in __exit__
    self.wait()
  File "/tmp/Bazel.runfiles_ab94s9vi/runfiles/cgr_fcs/apps/private/run_gx/run_gx.py", line 199, in wait
    assert num_errors == 0, "Had errors."
AssertionError: ESC[4mHad errors.ESC[0m

Here are the software versions used for this run: OS: Ubuntu 18.04.6 LTS Singularity: v3.10.0 FCS image: 0.5.0 Python: 3.8.12

Platform: LSF

Do you know what this error message means and how to prevent these crashes?

etvedte commented 4 months ago

Hello,

Fatal error (St13runtime_error): segment.cpp:506 in sort_and_merge(...) Do you have any >2 Gbp sequences in your FASTA files? This might be the source of the error. We do not have plans to support input sequences >2 Gbp.

Eric

eeaunin commented 4 months ago

Hello. Yes, this crash occurred with FASTA files that contained sequences larger than 2 Gb. I guess that explains it then