Oshlack / JAFFA

JAFFA is a multi-step pipeline that takes either raw RNA-Seq reads, or pre-assembled transcripts, then searches for gene fusions
https://github.com/Oshlack/JAFFA/wiki
Other
86 stars 21 forks source link

Pipeline fails at multiple stages on simulated nanopore dataset: mismatch between length of bases and qualities for reads #94

Open RDorney opened 1 year ago

RDorney commented 1 year ago

Hello, I am currently testing JAFFAL and other long-read fusion finding tools on a simulated dataset ~generated using badread to simulate reads and FUSIM to simulate fusions (including tri-fusions). The command I use:

/opt/JAFFA-version-2.3/tools/bin/bpipe run -n 3 /opt/JAFFA-version-2.3/JAFFAL.groovy ../*positive_control_nanopore2020_D*.fastq.gz                                                                                                 

The pipeline runs successfully and (relatively) smoothly on my spiked-in simulated pacbio dataset. except for this error message when entering Stage report_3_gene_fusions :

read id: fd35739b-5ecc-ca47-d0fd-05d771853c07 does not conform to 3 gene fusion structure.

(and I get this type of message for several reads in several data files). However, my main issue is when I run JAFFAL on my spiked-in simulated nanopore dataset where the pipeline fails, and I do not get a jaffa_results.csv or .fasta output file.

========================================= Pipeline Failed ==========================================
One or more parallel stages aborted. The following messages were reported:
Branch positive_control_nanopore2020_D100x_M90.fastq in stage Unknown reported message:
Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D100x_M90.fastq.gz out=positive_control_nanopore2020_D100x_M90.fastq/positive_control_nanopore2020_D100x_M90.fastq.fasta threads=1 ;
Branch positive_control_nanopore2020_D20x_M85.fastq in stage Unknown reported message:
Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D20x_M85.fastq.gz out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D20x_M85.fastq.fasta threads=1 ;
Branch positive_control_nanopore2020_D50x_M85.fastq in stage Unknown reported message:
Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D50x_M85.fastq.gz out=positive_control_nanopore2020_D50x_M85.fastq/positive_control_nanopore2020_D50x_M85.fastq.fasta threads=1 ;
Branch positive_control_nanopore2020_D50x_M90.fastq in stage Unknown reported message:
Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D50x_M90.fastq.gz out=positive_control_nanopore2020_D50x_M90.fastq/positive_control_nanopore2020_D50x_M90.fastq.fasta threads=1 ;
Use 'bpipe errors' to see output from failed commands.
ryleyd@minion:/data2/ryleyd/benchmark_simulated_reads/positive_control/nanopore$ /opt/JAFFA-version-2.3/tools/bin/bpipe errors
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass$3$1 (file:/opt/JAFFA-version-2.3/tools/bpipe-0.9.9.2/lib/bpipe.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass$3$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
============================== Found 4 failed commands from run 99149 ==============================
=========================================== Command 288 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D100x_M90.fastq.gz out=positive_control_nanopore2020_D100x_M90.fastq/positive_control_nanopore2020_D100x_M90.fastq.fasta threads=1 ;
Exit Code  : 1
Output     :
No command output found in most recent log file
=========================================== Command 297 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D20x_M85.fastq.gz out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D20x_M85.fastq.fasta threads=1 ;
Exit Code  : 1
Output     :
No command output found in most recent log file
=========================================== Command 352 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D50x_M85.fastq.gz out=positive_control_nanopore2020_D50x_M85.fastq/positive_control_nanopore2020_D50x_M85.fastq.fasta threads=1 ;
Exit Code  : 1
Output     :
No command output found in most recent log file
=========================================== Command 356 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D50x_M90.fastq.gz out=positive_control_nanopore2020_D50x_M90.fastq/positive_control_nanopore2020_D50x_M90.fastq.fasta threads=1 ;
Exit Code  : 1
Output     :
No command output found in most recent log file
RDorney commented 1 year ago

More information on the error messages I receive while the pipeline is running and files are progressing through stages

For several (but not all) files I will run into errors like:

=== Stage minimap2_transcriptome (positive_control_nanopore2020_D10x_M90) ===
Mismatch between length of bases and qualities for read 9276039 (id=2db111e8-9f27-5a08-a383-d2af7b6d92bd ENST00000391975.5|ENSG00000115677.1                                                                                                                                  8|OTTHUMG00000133391.10|OTTHUMT00000257245.6|HDLBP-203|HDLBP|6372|protein_coding|,+strand,0-6372 length=5861 error-free_length=6383 read_ide                                                                                                                                  ntity=78.63%).
# qualities=138, # bases=5861
....
        at stream.Read.validate(Read.java:114)
        at stream.Read.<init>(Read.java:78)
        at stream.Read.<init>(Read.java:61)
        at stream.FASTQ.quadToRead(FASTQ.java:872)
        at stream.FASTQ.toReadList(FASTQ.java:705)
        at stream.FastqReadInputStream.fillBuffer(FastqReadInputStream.java:111)
        at stream.FastqReadInputStream.nextList(FastqReadInputStream.java:96)
        at stream.ConcurrentGenericReadInputStream$ReadThread.readLists(ConcurrentGenericReadInputStream.java:656)
        at stream.ConcurrentGenericReadInputStream$ReadThread.run(ConcurrentGenericReadInputStream.java:635)
Input:                          9276000 reads           16463207847 bases
Output:                         9276000 reads (100.00%)         16463207847 bases (100.00%)
Time:                           251.239 seconds.
Reads Processed:       9276k    36.92k reads/sec
Bases Processed:      16463m    65.53m bases/sec
Exception in thread "main" java.lang.RuntimeException: ReformatReads terminated in an error state; the output may be corrupt.
        at jgi.ReformatReads.process(ReformatReads.java:1103)
        at jgi.ReformatReads.main(ReformatReads.java:43)
ERROR: Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D100x_M90.fastq.gz out=positive_control_nan                                                                                                                                  opore2020_D100x_M90.fastq/positive_control_nanopore2020_D100x_M90.fastq.fasta threads=1 ;

= Stage minimap2_transcriptome (positive_control_nanopore2020_D10x_M95) ~runs without error

== Stage filter_transcripts (positive_control_nanopore2020_D10x_M95) ==
java.lang.AssertionError:
Mismatch between length of bases and qualities for read 1934044 (id=d642ebd4-a024-95da-520b-1b29b14a4348 ENST00000361624.2|ENSG00000198804.2|-|-|MT-CO1-201|MT-CO1|1542|protein_coding|,-strand,0-1542 chimera ENST00000582467.2|ENSG000001716                                34.19|OTTHUMG00000132254.6|OTTHUMT00000448091.2|BPTF-215|BPTF|11789|protein_coding|,+strand,0-11789 length=12759 error-free_length=13347 read_identity=87.51%).
# qualities=12894, # bases=12759
at stream.Read.validate(Read.java:114)
        at stream.Read.<init>(Read.java:78)
        at stream.Read.<init>(Read.java:61)
        at stream.FASTQ.quadToRead(FASTQ.java:872)
        at stream.FASTQ.toReadList(FASTQ.java:705)
        at stream.FastqReadInputStream.fillBuffer(FastqReadInputStream.java:111)
        at stream.FastqReadInputStream.nextList(FastqReadInputStream.java:96)
        at stream.ConcurrentGenericReadInputStream$ReadThread.readLists(ConcurrentGenericReadInputStream.java:656)
        at stream.ConcurrentGenericReadInputStream$ReadThread.run(ConcurrentGenericReadInputStream.java:635)
Input:                          1934000 reads           3369013255 bases
Output:                         1934000 reads (100.00%)         3369013255 bases (100.00%)
Time:                           50.653 seconds.
Reads Processed:       1934k    38.18k reads/sec
Bases Processed:       3369m    66.51m bases/sec
Exception in thread "main" java.lang.RuntimeException: ReformatReads terminated in an error state; the output may be corrupt.
        at jgi.ReformatReads.process(ReformatReads.java:1103)
        at jgi.ReformatReads.main(ReformatReads.java:43)
ERROR: Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D20x_M85.fastq.gz out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D20x_M85.fastq.fasta threads=1 ;
================ Stage filter_transcripts (positive_control_nanopore2020_D10x_M95) =================
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads ignorebadquality=t in=../positive_control_nanopore2020_D20x_M85.fastq.gz out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D2                                0x_M85.fastq.fasta threads=1
Executing jgi.ReformatReads [ignorebadquality=t, in=../positive_control_nanopore2020_D20x_M85.fastq.gz, out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D20x_M85.fastq.fasta, threads=1]
Set threads to 1
Input is being processed as unpaired
[M::worker_pipeline::3857.303*1.00] mapped 286876 sequences
java.lang.AssertionError:
Mismatch between length of bases and qualities for read 1934044 (id=d642ebd4-a024-95da-520b-1b29b14a4348 ENST00000361624.2|ENSG00000198804.2|-|-|MT-CO1-201|MT-CO1|1542|protein_coding|,-strand,0-1542 chimera ENST00000582467.2|ENSG000001716                                34.19|OTTHUMG00000132254.6|OTTHUMT00000448091.2|BPTF-215|BPTF|11789|protein_coding|,+strand,0-11789 length=12759 error-free_length=13347 read_identity=87.51%).
# qualities=12894, # bases=12759
......
 at stream.Read.validate(Read.java:114)
        at stream.Read.<init>(Read.java:78)
        at stream.Read.<init>(Read.java:61)
        at stream.FASTQ.quadToRead(FASTQ.java:872)
        at stream.FASTQ.toReadList(FASTQ.java:705)
        at stream.FastqReadInputStream.fillBuffer(FastqReadInputStream.java:111)
        at stream.FastqReadInputStream.nextList(FastqReadInputStream.java:96)
        at stream.ConcurrentGenericReadInputStream$ReadThread.readLists(ConcurrentGenericReadInputStream.java:656)
        at stream.ConcurrentGenericReadInputStream$ReadThread.run(ConcurrentGenericReadInputStream.java:635)
Input:                          1934000 reads           3369013255 bases
Output:                         1934000 reads (100.00%)         3369013255 bases (100.00%)
Time:                           50.653 seconds.
Reads Processed:       1934k    38.18k reads/sec
Bases Processed:       3369m    66.51m bases/sec
Exception in thread "main" java.lang.RuntimeException: ReformatReads terminated in an error state; the output may be corrupt.
        at jgi.ReformatReads.process(ReformatReads.java:1103)
        at jgi.ReformatReads.main(ReformatReads.java:43)
ERROR: Command failed with exit status = 1 :
/opt/JAFFA-version-2.3/tools/bin/reformat ignorebadquality=t in=../positive_control_nanopore2020_D20x_M85.fastq.gz out=positive_control_nanopore2020_D20x_M85.fastq/positive_control_nanopore2020_D20x_M85.fastq.fasta threads=1 ;

Despite these errors, the files will continue into later stages:

============= Stage extract_fusion_sequences (positive_control_nanopore2020_D10x_M95) ==============
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads ignorebadquality=t in=../positive_control_nanopore2020_D20x_M90.fastq.gz out=positive_control_nanopore2020_D20x_M90.fastq/positive_control_nanopore2020_D20x_M90.fastq.fasta threads=1
Executing jgi.ReformatReads [ignorebadquality=t, in=../positive_control_nanopore2020_D20x_M90.fastq.gz, out=positive_control_nanopore2020_D20x_M90.fastq/positive_control_nanopore2020_D20x_M90.fastq.fasta, threads=1]
Set threads to 1
Input is being processed as unpaired
[M::worker_pipeline::4170.854*1.00] mapped 207573 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x map-ont -c /opt/JAFFA-version-2.3/hg38_genCode22.fa positive_control_nanopore2020_D10x_M90.fastq/positive_control_nanopore2020_D10x_M90.fastq.fasta
[M::main] Real time: 4170.911 sec; CPU: 4170.349 sec; Peak RSS: 2.835 GB
================ Stage filter_transcripts (positive_control_nanopore2020_D10x_M90) =================
Done reading in transcript IDs
Reading the input alignment file, jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq/jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq.paf
0
1000000
2000000
3000000
3038753 reads processed. Finished.
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads in=jaffal_edited_positive_control_nanopore2020_D10x_M85.fastq/jaffal_edited_positive_control_nanopore2020_D10x_M85.fastq.fasta out=stdout.fasta fastawrap=0
....
============= Stage extract_fusion_sequences (positive_control_nanopore2020_D10x_M90) ==============
Executing jgi.ReformatReads [in=jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq/jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq.fasta, out=stdout.fasta, fastawrap=0]
Input is being processed as unpaired
Input:                          2418119 reads           4376157978 bases
Output:                         2418119 reads (100.00%)         4376157978 bases (100.00%)
Time:                           74.634 seconds.
Reads Processed:       2418k    32.40k reads/sec
Bases Processed:       4376m    58.64m bases/sec
================== Stage minimap2_genome (positive_control_nanopore2020_D10x_M95) ==================
[M::mm_idx_gen::11.980*0.83] collected minimizers
[M::mm_idx_gen::16.631*0.88] sorted minimizers
[M::main::16.631*0.88] loaded/built the index for 195175 target sequence(s)
[M::mm_mapopt_update::17.107*0.88] mid_occ = 110
[M::mm_idx_stat] kmer size: 15; skip: 10; is_hpc: 0; #seq: 195175
[M::mm_idx_stat::17.405*0.88] distinct minimizers: 16950829 (43.60% are singletons); average occurrences: 3.243; average spacing: 5.406
[M::worker_pipeline::3316.073*1.00] mapped 276210 sequences
Input:                          3432565 reads           6048616768 bases
Output:                         3432565 reads (100.00%)         6048616768 bases (100.00%)
Time:                           104.254 seconds.
Reads Processed:       3432k    32.92k reads/sec
Bases Processed:       6048m    58.02m bases/sec
================== Stage minimap2_genome (positive_control_nanopore2020_D10x_M90) ==================
[M::mm_idx_gen::129.049*0.96] collected minimizers
[M::worker_pipeline::961.815*1.00] mapped 282471 sequences
[M::worker_pipeline::4260.013*1.00] mapped 276724 sequences
[M::mm_idx_gen::217.976*0.98] sorted minimizers
[M::main::217.976*0.98] loaded/built the index for 455 target sequence(s)
[M::mm_mapopt_update::221.762*0.98] mid_occ = 792
[M::mm_idx_stat] kmer size: 15; skip: 5; is_hpc: 0; #seq: 455
[M::mm_idx_stat::224.110*0.98] distinct minimizers: 167291034 (34.68% are singletons); average occurrences: 6.239; average spacing: 3.075
[M::worker_pipeline::1435.168*1.00] mapped 281994 sequences
[M::worker_pipeline::4730.434*1.00] mapped 276264 sequences
[M::worker_pipeline::1019.761*0.99] mapped 12621 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa positive_control_nanopore2020_D10x_M95.fastq/positive_control_nanopore2020_D10x_M95.fastq.fusions.fa
[M::main] Real time: 1019.905 sec; CPU: 1014.720 sec; Peak RSS: 18.983 GB
============== Stage make_fasta_reads_table (positive_control_nanopore2020_D10x_M95) ===============
[M::worker_pipeline::1907.097*1.00] mapped 282043 sequences
[M::worker_pipeline::5201.252*1.00] mapped 276573 sequences
[M::mm_idx_gen::144.855*0.86] collected minimizers
[M::mm_idx_gen::233.550*0.91] sorted minimizers
[M::main::233.550*0.91] loaded/built the index for 455 target sequence(s)
[M::mm_mapopt_update::237.213*0.92] mid_occ = 792
[M::mm_idx_stat] kmer size: 15; skip: 5; is_hpc: 0; #seq: 455
[M::mm_idx_stat::239.559*0.92] distinct minimizers: 167291034 (34.68% are singletons); average occurrences: 6.239; average spacing: 3.075
[M::worker_pipeline::2381.133*1.00] mapped 281944 sequences
[M::worker_pipeline::5674.755*1.00] mapped 276818 sequences
[M::worker_pipeline::2854.149*1.00] mapped 281181 sequences
[M::worker_pipeline::6145.728*1.00] mapped 276402 sequences
[M::worker_pipeline::3327.070*1.00] mapped 281324 sequences
[M::worker_pipeline::6616.826*1.00] mapped 276139 sequences
[M::worker_pipeline::3799.048*1.00] mapped 282279 sequences
[M::worker_pipeline::7087.283*1.00] mapped 275430 sequences
[M::worker_pipeline::2286.720*0.99] mapped 25347 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq/jaffal_edited_positive_control_nanopore2020_D100x_M85.fastq.fusions.fa
[M::main] Real time: 2286.867 sec; CPU: 2266.908 sec; Peak RSS: 18.982 GB
============== Stage make_fasta_reads_table (positive_control_nanopore2020_D10x_M90) ===============
================== Stage get_final_list (positive_control_nanopore2020_D10x_M95) ===================
================== Stage get_final_list (positive_control_nanopore2020_D10x_M90) ===================
R version 4.3.0 (2023-04-21) -- "Already Tomorrow"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
  Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> options(echo=FALSE)
[1] "Getting the location of fusion transcripts in the genome.."
Calculating gap size in the genome...
Checking if the fusions are in frame...
[M::worker_pipeline::6645.681*1.00] mapped 281746 sequences
[M::worker_pipeline::9932.681*1.00] mapped 276604 sequences
Merging with read coverage data...
Reassigning Low Confidence breakpoints
Done producing summary file
=============== Stage report_3_gene_fusions (positive_control_nanopore2020_D10x_M95) ===============
read id: f0b35753-71f3-f9c3-9a50-77dbb994040b does not conform to 3 gene fusion structure.

However I then get the same error message again in

== Stage report_3_gene_fusions (positive_control_nanopore2020_D10x_M90) ==
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads ignorebadquality=t in=../positive_control_nanopore2020_D50x_M85.fastq.gz out=positive_control_nanopore2020_D50x_M85.fastq/positive_control_nanopore2020_D50x_M85.fastq.fasta threads=1
Executing jgi.ReformatReads [ignorebadquality=t, in=../positive_control_nanopore2020_D50x_M85.fastq.gz, out=positive_control_nanopore2020_D50x_M85.fastq/positive_control_nanopore2020_D50x_M85.fastq.fasta, threads=1]
Set threads to 1
Input is being processed as unpaired
java.lang.AssertionError:
Mismatch between length of bases and qualities for read 2524899 (id=8f66a3fe-c408-fef8-1779-6ce91cc4a34f ENST00000344063.7|ENSG00000108107.15|OTTHUMG00000171991.4|OTTHUMT00000416277.2|RPL28-201|RPL28|4209|protein_coding|,+strand,0-4209 length=4170 error-free_length=4235 read_identity=94.79%).
# qualities=135, # bases=4170
@c9990bd9-8607-294c-8734-feb361710e64 ref|NM_018666-NM_001330000,+strand,0-5484 length=5419 error-free_length=5485 read_identity=89.20%
CGTTCAGTTACGTATTGCTCTCTTTCCGGTCATCAGGTCGCCGCTGCGAAGGGGCCGCCACCATGTCTGCGCATCTGCAATGGATGGTCGTGCGGAACTGCTCCAGTTTCCTGATCAACGAGGAATAAGCAGACCTACAGCACTGAGCCCAATAACTTGAAGGCCCGCAATTCAACTTCCGCTACAGCGGATGATTCACGCAGGACTGTGGGCGTGGAGCCGGCGGCCGACGGCAAAGTGTCGTGGTGGTCATTAAGCAGATCCGGCCAGCGGAAGCCTGCCACCTCCTATGTGCGGACACCATCAACAAGAATGCTCGCGCCCACGCTCAGCAGCATCAGACACATGATCCGCAAGAACAAGTACCGCCCCGACCTGCTGATGGCAGCCATCCGCAGGGCCAGCGCCATCCTGTGCAGCCAGAAGCCTGTGATGGTGAAGAAAGCGGACCGCCCCACCAAGAGCTCCTGAGCCCCCTGCCCCCAGAGCAATAAAGTCAGCTGCTTTCTCACCCTGCCTCGACGGGCCTCCCTTTTTTGAAACGCTCTGGGGAGCTCTGGCCTATGTTGTCATTCAGGCCATGTCATCAAACTCTGCATGTCACCTTGTCCATCTGGAGGTGATGTCAATGGCTGGCCACATGCAGGAGGGGTGGGGTAGCTGCCTTGTCCTTTGGTGGCAAGGGTCACTGTCTTCACAGAAAAATTTGCTGACTTGTGATTGAACCTACTGTCCCATTGTGAGGTGGCCTGAAGAATCCCAGCTGGGGAGTGGCTTCCATTCAGAAGAAGAAAGGCCTTTTCTAGCCCAGAAGGGTGCAGGCTGAGGCTGGGCCCTGGGCCCTGGTGCTGTAGCACGGTTTGGGGACTTGGGGTGTTCCCAAGACCTGGGGGACGACGAACATCCGGGAGGAAGATGAGAGACTTTTGCATCCAGGAGTGGGTGCGGCCACATTTGGAGGGGATGGAGCTTTGATGCAACCTGTCTCTGAGATGGGCAACTTTGGTGGGTGGTGGCTTATAACTGTAAGGGGATGGCTGACCAGGGTACAGCCAGCAGGCATTGAGCAGCCTTAGCATTGTCCCCCTACTCCCGTCCTCCAGGTGTCCGCATCCCTCCCCTATCTCTTTGAGCTGGCTCTTGTCACCTTAGGACTCATCTAGTGGCCGCTCTTTGGGCCACCCTGTCACCCAAGCTTTCCTGATTGCCCAGCCCTCTTGTTTCCTTTGGCCTGTTTGCTCCCTAGTGTTTATTACAGCTTGTGAGGCCAGGAGTTTGAGACCATCCTAGGCAACACCAATGAGACACCGTCTCTAAAATAAAATTAGCTGGGTGTGGTGGTCACCGCCTGTGGTCCCAGCTCCTCAGAGGTTGAGTGGGGCTGAGGGACGGAGCACTTGAACCAAGAATATGGGGCTGCAGTAGCCCAGAGCCCCACCACCACACTCCAGCCTGGAAGACACCATGACACAGTGAGGCCTGGATGGAAAAGAGTCCTGTGTTGATCCTCACATGTTTCCTGAACACTAACTCTGTCAGCCACTGCCAGGACCAAAGGATCAGCATCATGGCACCCCTGGTTCCACTGCCATCCTGGGGGTACCGATTCAAAGAAGGACTCTGCTCCCTGTCTGAGACCACCCCCCGGCTCTGACTGAGATAAGGGGACTGTCAGGGCCTCGACTTGCCATTGGTTGGGGTCGTACGGGGCTGGGAGCCTGCGTTTTGAGGCGGGCCTGCCCTTCCGACCTCAGTCCTGTCTGCTCCAGTCTTGCCCAACTCGAAGGGAGCAGATCTGACCACTTGCCAGCCCCTGTCTGCTGTGAATTACATTTCCTTTGTCTTCCTTAGTTGGGTCTATTAGCTCAGATTGAGGTGTTACCTTAAAATGAGTTGGGTGACTTGGTACCTGCTCAGGACTCCCCCGCACTGTCCGTCCCACTCAGGCCCACCTCCAGCTGGCCTCACTCCGCTGAGTGACTTCGTACCTGCTCAGGAGCCCCCACTGTCCCAGTCCCACTCAGGCCCATCTCGCTGGCCTCACTGCGCTGGGACTCCGCCTTCATAAAGGAGAGCTCACTGCTCACGTTAGTAAGATGGCCCCTTCTCGTGAGGCCTCTCCCTGGCACCTGCTTCAGTTGTCCTCCACAGCACTGATTTGCAGCCCACAAGCTGGCAGGTTTATCTGTCTCATGTTTGTCTTGTGCTGGTCGGCAAGGGGTTTAGTCTAGCACACCAGCATATAATGAGATGCTTGATGAATGGTACATATTGAATGTATAAAGCCCACCGGTCCTGAGTTTGCTCACTGGAGACTTTCTGGAGATGGAGTCTCGCTCTGTGCCCGGCTGGCGAAGTGCAATGGCGCGATCTTGGCTCACTGCAGCCTCCCACCTCCTGGGTTCAACGATTCTCCTGCCTCAGCCTCCCGAGTAGCTGGGATTACAGGTGGTATGCCACACCCAGCTCAGTATTGTATTTTTAGCAGAGATCGGGTTTCACCATTGCAGGCTGGTTTGGAACTCCTGACTTCAAATTACCCACCTGCCTCAACCTCCCAAAGTGCTGGCATTACAAGCGCTCGAGGCTTCTGATGTGGCTGCTGCTGCTCAGAAGGCCTTGTCCTTAACCACCTCCTTGCCTGCCCTGGAGGCTTGTGCCTCTAGGACCCCACCCCTGTGGAGTCCTGCTGGCTTTCTCCATCCCTATCTGAATCCTCCTGCTGTGTGGCCTCCCCTGGTCTCATCCGTAACACAGCCCAACAGTGGCCTCTGTTCCTGCGGGTGGCCAGCCCATCTGTGTGTGGCTGGGCTGGGGAGGCCACGCGTCTGGTATCTGAATGCTATCGGTGGGTTGGGGTGGAGGAACCAGGAGAGGGCTGGAGGGAGAGGAGATGGTCTCAGCCCCACAGAGTTTGGATGTCATCAGTGTGCTGAGCAAACGTGAACACCATTTCCCTCCTCTAGACCTCATCTTGGAGAGAGAGATGTTGGATGGGGCCATCTATTCCAGCTTTATTCACAAATCATGTCTGTTGGCCTGGAAATTGGAAAACCAGTTAAACCAAAAACGTGATATTAAGAAAACAGGCGGAGCTCACCATTGTAAAAATGCTGAAAGCCAAAGACAAAGTGGAGAACAAAGAAAAGCGTCTTGTCACATTACAGAAGGTCCTGATAAAGTAGTAGCTGCCCTCATCTAGAAACCAGGCCCAGGCAGTGGGGACACATCCAGTGCTGAAAGAACCTCCCCCAGGTCATCTATCCCAAGAGTGATGCCCGGCAGCATTCCAAGCTCAAGGGCTAATGGTTCACGGAAGCCAGGAATCAAACTGCCTGGGTTCCAGTCCCAGCTCTGCCAGTTATGCCCAGCTGTGGGACTTGGGCAACTCGTTTAGTAGCACCGTGCCTCAGTTCCCAATATGTAAAAGGCCATTTGAGTGCCTTTCACAGCCCTGCATAAGGCAGGTGTCTCAGTGTTCACTGCTATCTCTCCAGCTCTTAGTCCAGTAGCTACATGGTGAGTGAGCGTAGGGCGCACCCTGGAAGGCTGCCCAGCCCAAAGTTGTGCCAGAGCGCTGGGGATCAGACTCCCCACAGCAGCAGAGACTCAGGGACTGAGGCATCCTCTGTCACAGGACATGCTGCGTCTACTGGGTCAGGGCTCTGCTGCTCGGTGGCTGTGAACCTTAGGCAAGTTCCTCAACCTCTCTGTGTCTTCGTACCCTCATCTGTAACATGCGTGTCGATAGACCCTACTACTCCGGGTTGGTGAAGATTAAATGTGCAAAACCTGCTTGACGCTGTGCCCACAAATCCTGATTGTAGGAATAAATTAATGACTTTTTATAAATATTTTGATCGATGGACTCATGATCTGGATGTCTCACATGCCAATGACTAATTTGTACACAAACTAATGCTCGTGTTTCCCGAGCACCTGGAGAGACATAGCCAGATCCATGTGCGATAATGCCTGGTGGCTCCAGGTTGCCCCGCCGTCCTGTGGGCTGTGAGCTTTCCCAGCCTCCTGCCCGTGTTTGTGAATATCATTCTGTCCTCCAGCTGCATTTCCAGCCGAGCTCTTTGGCGCTGCCCAGGAATGGTATCAATTCCCCTGTTTCTCTGTAGCCAGTTACTAGAATAAAATCATCTACTTTAAAAGCAATAC
        at stream.Read.validate(Read.java:114)
        at stream.Read.<init>(Read.java:78)
        at stream.Read.<init>(Read.java:61)
        at stream.FASTQ.quadToRead(FASTQ.java:872)
        at stream.FASTQ.toReadList(FASTQ.java:705)
        at stream.FastqReadInputStream.fillBuffer(FastqReadInputStream.java:111)
        at stream.FastqReadInputStream.nextList(FastqReadInputStream.java:96)
        at stream.ConcurrentGenericReadInputStream$ReadThread.readLists(ConcurrentGenericReadInputStream.java:656)
        at stream.ConcurrentGenericReadInputStream$ReadThread.run(ConcurrentGenericReadInputStream.java:635)
Input:                          2524800 reads           4397913090 bases
Output:                         2524800 reads (100.00%)         4397913090 bases (100.00%)
Time:                           66.580 seconds.
Reads Processed:       2524k    37.92k reads/sec
Bases Processed:       4397m    66.05m bases/sec
Exception in thread "main" java.lang.RuntimeException: ReformatReads terminated in an error state; the output may be corrupt.
        at jgi.ReformatReads.process(ReformatReads.java:1103)
        at jgi.ReformatReads.main(ReformatReads.java:43)
read id: 97bbcd5f-5c99-c079-08c5-f9de492e7062 does not conform to 3 gene fusion structure.
read id: 0ed90292-a03b-79f8-49c7-a0f6cad8a986 does not conform to 3 gene fusion structure.
read id: a8a332ef-bc95-08a1-ff04-adc08e25bde0 does not conform to 3 gene fusion structure.
read id: 785523d8-f840-1378-82ab-088fd95db875 does not conform to 3 gene fusion structure.
read id: 7ed251aa-ab58-b50a-fab5-1d65e4bc61f6 does not conform to 3 gene fusion structure.
read id: 13001a1f-1ef1-115e-16e8-6b80f23c960c does not conform to 3 gene fusion structure.
read id: 26a7d451-1849-ee01-4ef4-7e5b4f9731b4 does not conform to 3 gene fusion structure.
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads ignorebadquality=t in=../positive_control_nanopore2020_D20x_M95.fastq.gz out=positive_control_nanopore2020_D20x_M95.fastq/positive_control_nanopore2020_D20x_M95.fastq.fasta threads=1
RDorney commented 1 year ago

Running JAFFAL on only the spiked fusions, and only the background simulated reads

In an attempt to narrow down what may be cause the errors, I tried run JAFFAL on on only the spiked fusions, and only the background simulated reads (these files are appended to each other to make my "positive_control" data files I ran JAFFAL on earlier. The pipeline fails, but with different exit statuses and error messages background reads, no spiked fusions

========================================= Pipeline Failed ==========================================
One or more parallel stages aborted. The following messages were reported:
Branch negative.control.nanopore2020_D10x_M90_HepG2.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf;          grep $'\t+\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ;           grep $'\t-\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ;
Branch negative.control.nanopore2020_D10x_M95_HepG2.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 5 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf;          grep $'\t+\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ;           grep $'\t-\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ;
Use 'bpipe errors' to see output from failed commands.
ryleyd@minion:/data2/ryleyd/benchmark_simulated_reads/no_fusions/troubleshooting$ /opt/JAFFA-version-2.3/tools/bin/bpipe errors
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass$3$1 (file:/opt/JAFFA-version-2.3/tools/bpipe-0.9.9.2/lib/bpipe.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass$3$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
============================= Found 2 failed commands from run 488499 ==============================
============================================ Command 24 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 5 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf;     grep $'\t+\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ;           grep $'\t-\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ;
Exit Code  : 137
Output     :
No command output found in most recent log file
============================================ Command 25 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf;     grep $'\t+\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ;           grep $'\t-\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ;
Exit Code  : 137
Output     :
No command output found in most recent log file

Other errors when JAFFAL is run only on background reads

========== Stage extract_fusion_sequences (negative.control.nanopore2020_D10x_M85_HepG2) ===========
bash: line 1: 491905 Killed                  /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf
java -ea -Xmx200m -cp /opt/JAFFA-version-2.3/tools/bbmap/current/ jgi.ReformatReads in=negative.control.nanopore2020_D10x_M85_HepG2.fastq/negative.control.nanopore2020_D10x_M85_HepG2.fastq.fasta out=stdout.fasta fastawrap=0
Executing jgi.ReformatReads [in=negative.control.nanopore2020_D10x_M85_HepG2.fastq/negative.control.nanopore2020_D10x_M85_HepG2.fastq.fasta, out=stdout.fasta, fastawrap=0]
Input is being processed as unpaired
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf;          grep $'\t+\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ;           grep $'\t-\t' negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M90_HepG2.fastq/negative.control.nanopore2020_D10x_M90_HepG2.fastq_genome.psl ;
Input:                          1727872 reads           3009507128 bases
Output:                         1727872 reads (100.00%)         3009507128 bases (100.00%)
Time:                           50.819 seconds.
Reads Processed:       1727k    34.00k reads/sec
Bases Processed:       3009m    59.22m bases/sec
=============== Stage minimap2_genome (negative.control.nanopore2020_D10x_M85_HepG2) ===============
[M::mm_idx_gen::781.869*0.20] collected minimizers
bash: line 1: 491895 Killed                  /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 5 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 5 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq.fusions.fa > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf;          grep $'\t+\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ;           grep $'\t-\t' negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> negative.control.nanopore2020_D10x_M95_HepG2.fastq/negative.control.nanopore2020_D10x_M95_HepG2.fastq_genome.psl ;
[M::mm_idx_gen::701.496*0.21] collected minimizers
[M::mm_idx_gen::1218.171*0.29] sorted minimizers
[M::main::1218.171*0.29] loaded/built the index for 455 target sequence(s)
[M::mm_mapopt_update::1221.510*0.29] mid_occ = 792
[M::mm_idx_stat] kmer size: 15; skip: 5; is_hpc: 0; #seq: 455
[M::mm_idx_stat::1223.453*0.29] distinct minimizers: 167291034 (34.68% are singletons); average occurrences: 6.239; average spacing: 3.075
[M::worker_pipeline::1414.630*0.39] mapped 4878 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa negative.control.nanopore2020_D10x_M85_HepG2.fastq/negative.control.nanopore2020_D10x_M85_HepG2.fastq.fusions.fa
[M::main] Real time: 1414.818 sec; CPU: 549.583 sec; Peak RSS: 18.681 GB

Only fusion reads, no background reads

========================================= Pipeline Failed ==========================================
One or more parallel stages aborted. The following messages were reported:
Branch nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq.fusions.fa > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf;      grep $'\t+\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ;       grep $'\t-\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ;
Use 'bpipe errors' to see output from failed commands.
ryleyd@minion:/data2/ryleyd/benchmark_simulated_reads/fusions/JAFFAL_troubleshooting$ /opt/JAFFA-version-2.3/tools/bin/bpipe errors
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass$3$1 (file:/opt/JAFFA-version-2.3/tools/bpipe-0.9.9.2/lib/bpipe.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass$3$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

============================= Found 1 failed commands from run 488798 ==============================
============================================ Command 27 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq.fusions.fa > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf;         grep $'\t+\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ;       grep $'\t-\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ;
Exit Code  : 137
Output     :
No command output found in most recent log file

However, the only error that appears while running the code is as below:

============== Stage minimap2_genome (nanopore2020_D10x_M85_benchmark_fusions.fasta) ===============
[M::mm_idx_gen::86.966*1.57] collected minimizers
[M::mm_idx_gen::272.588*0.52] collected minimizers
[M::mm_idx_gen::412.544*0.35] collected minimizers
[M::mm_idx_gen::2049.733*0.13] sorted minimizers
[M::main::2049.754*0.13] loaded/built the index for 455 target sequence(s)
[M::mm_mapopt_update::2131.577*0.12] mid_occ = 792
[M::mm_idx_stat] kmer size: 15; skip: 5; is_hpc: 0; #seq: 455
[M::mm_idx_stat::2133.926*0.12] distinct minimizers: 167291034 (34.68% are singletons); average occurrences: 6.239; average spacing: 3.075
bash: line 1: 489181 Killed                  /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq.fusions.fa > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq.fusions.fa > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf;      grep $'\t+\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ;       grep $'\t-\t' nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M90_benchmark_fusions.fasta.fastq_genome.psl ;
[M::worker_pipeline::3764.259*0.18] mapped 2622 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 4 -x splice -c /opt/JAFFA-version-2.3/hg38.fa nanopore2020_D10x_M95_benchmark_fusions.fasta.fastq/nanopore2020_D10x_M95_benchmark_fusions.fasta.fastq.fusions.fa
[M::main] Real time: 3777.054 sec; CPU: 680.451 sec; Peak RSS: 19.134 GB
nadiadavidson commented 1 year ago

Hello,

Thanks for trying out JAFFAL and posting these issues. These don't seem familiar to me, so would you mind addressing the questions detailed here, https://github.com/Oshlack/JAFFA/wiki/FAQandTroubleshooting#other-errors so I can get some more insight into what might be happening. In particular if you can email a reproducible example that would be a great help.

One of the error is about the bases and quality lengths being different for some reads. Did you check if this was indeed the case in your simulated data as that could be the cause of the failure?

Cheers, Nadia.

RDorney commented 1 year ago

Thank you for getting back to me so quickly 😊 I ran JAFFAL on the simulated LongReadFusionSimulation Dataset and had some errors appear

======================== Stage extract_fusion_sequences (ONT_fus_sim_75err) ========================
bash: line 1: 32309 Killed                  /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq.fusions.fa > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq.fusions.fa > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf;        grep $'\t+\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ;
Done reading in transcript IDs
Reading the input alignment file, ONT_fus_sim_80err.fastq/ONT_fus_sim_80err.fastq.paf
0
15697 reads processed. Finished.
Done reading in transcript IDs
Reading the input alignment file, Pac_fus_sim_95err.fastq/Pac_fus_sim_95err.fastq.paf
============================ Stage minimap2_genome (Pac_fus_sim_95err) =============================
bash: line 1: 32315 Killed                  /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq.fusions.fa > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf
[M::mm_idx_gen::629.794*0.24] collected minimizers
bash: line 1: 32396 Killed                  /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq.fusions.fa > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq.fusions.fa > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf;        grep $'\t+\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ;       grep $'\t-\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ;
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq.fusions.fa > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf;        grep $'\t+\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ;
bash: line 1: 32326 Killed                  /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq.fusions.fa > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf
bash: line 1: 32633 Killed                  /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq.fusions.fa > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf
[M::mm_idx_gen::944.330*0.18] collected minimizers
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq.fusions.fa > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf;        grep $'\t+\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ;
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq.fusions.fa > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf;        grep $'\t+\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ;       grep $'\t-\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ;
bash: line 1: 32332 Killed                  /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq.fusions.fa > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq.fusions.fa > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf;        grep $'\t+\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ;
[M::mm_idx_gen::417.599*0.33] collected minimizers
[M::mm_idx_gen::374.296*0.40] collected minimizers
[M::mm_idx_gen::2238.580*0.11] collected minimizers
bash: line 1: 32380 Killed                  /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq.fusions.fa > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf
ERROR: Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq.fusions.fa > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf;        grep $'\t+\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ;       grep $'\t-\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ;
[M::mm_idx_gen::2702.171*0.11] collected minimizers
[M::mm_idx_gen::3319.178*0.12] sorted minimizers
[M::main::3319.178*0.12] loaded/built the index for 455 target sequence(s)
[M::mm_mapopt_update::3646.555*0.11] mid_occ = 792
[M::mm_idx_stat] kmer size: 15; skip: 5; is_hpc: 0; #seq: 455
[M::mm_idx_stat::3648.542*0.11] distinct minimizers: 167291034 (34.68% are singletons); average occurrences: 6.239; average spacing: 3.075
[M::worker_pipeline::5154.300*0.26] mapped 15855 sequences
[M::main] Version: 2.17-r941
[M::main] CMD: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_95err.fastq/Pac_fus_sim_95err.fastq.fusions.fa
[M::main] Real time: 5154.740 sec; CPU: 1343.433 sec; Peak RSS: 18.498 GB
========================================= Pipeline Failed ==========================================
One or more parallel stages aborted. The following messages were reported:
Branch ONT_fus_sim_85err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq.fusions.fa > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf;        grep $'\t+\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ;       grep $'\t-\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ;
Branch ONT_fus_sim_90err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq.fusions.fa > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf;        grep $'\t+\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ;       grep $'\t-\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ;
Branch ONT_fus_sim_95err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq.fusions.fa > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf;        grep $'\t+\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ;       grep $'\t-\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ;
Branch Pac_fus_sim_75err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq.fusions.fa > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf;        grep $'\t+\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ;
Branch Pac_fus_sim_80err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq.fusions.fa > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf;        grep $'\t+\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ;
Branch Pac_fus_sim_85err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq.fusions.fa > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf;        grep $'\t+\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ;
Branch Pac_fus_sim_90err.fastq in stage Unknown reported message:
Command failed with exit status = 137 :
/opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq.fusions.fa > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf;        grep $'\t+\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ;
Use 'bpipe errors' to see output from failed commands.
ryleyd@minion:/data2/ryleyd/benchmark_simulated_reads/JAFFAL_troubleshooting$ /opt/JAFFA-version-2.3/tools/bin/bpipe errors
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass$3$1 (file:/opt/JAFFA-version-2.3/tools/bpipe-0.9.9.2/lib/bpipe.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass$3$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
============================== Found 7 failed commands from run 31354 ==============================
============================================ Command 64 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq.fusions.fa > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf;           grep $'\t+\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.psl ;
Exit Code  : 137
Output     :
No command output found in most recent log file
============================================ Command 56 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq.fusions.fa > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf;           grep $'\t+\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ;       grep $'\t-\t' ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_95err.fastq/ONT_fus_sim_95err.fastq_genome.psl ;
Exit Code  : 137
Output     :
No command output found in most recent log file
============================================ Command 62 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq.fusions.fa > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf;           grep $'\t+\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_90err.fastq/Pac_fus_sim_90err.fastq_genome.psl ;
Exit Code  : 137
Output     :
No command output found in most recent log file
============================================ Command 60 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq.fusions.fa > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf;           grep $'\t+\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_75err.fastq/Pac_fus_sim_75err.fastq_genome.psl ;
Exit Code  : 137
Output     :
No command output found in most recent log file
============================================ Command 66 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq.fusions.fa > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf;           grep $'\t+\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ;       grep $'\t-\t' ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_85err.fastq/ONT_fus_sim_85err.fastq_genome.psl ;
Exit Code  : 137
Output     :
No command output found in most recent log file
============================================ Command 74 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq.fusions.fa > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf;           grep $'\t+\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ;       grep $'\t-\t' Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> Pac_fus_sim_85err.fastq/Pac_fus_sim_85err.fastq_genome.psl ;
Exit Code  : 137
Output     :
No command output found in most recent log file
============================================ Command 86 ============================================
Command    : /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq.fusions.fa > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf;           grep $'\t+\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",",$3","$3+100",",  $8","$9-$4+$3+100"," }' > ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ;       grep $'\t-\t' ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.paf | awk -F'\t' -v OFS="\t" '{ print $4-$3,0,0,0,0,0,0,0,$5,$1,$2,$3,$4,$6,$7,$8,$9,2, 100","$4-$3-100",", $2-$4","$2-$4+100",", $8","$9-$4+$3+100"," }' >> ONT_fus_sim_90err.fastq/ONT_fus_sim_90err.fastq_genome.psl ;
Exit Code  : 137
Output     :
No command output found in most recent log file

More information on directory contents

ryleyd@minion:~$ ls -lh /opt/JAFFA-version-2.3/
total 13G
-rwxrwxr-x  1 root       root       2.3K Jul 24  2022 assemble.sh
-rwxrwxr-x  1 root       root       2.5K Jul 24  2022 compile_results.R
-rw-rw-r--  1 root       root        35K Jul 24  2022 COPYING
drwxrwxr-x  2 root       root       4.0K Jul 24  2022 cwl
drwxrwxr-x  2 root       root       4.0K Jul 24  2022 docker
-rwxrwxr-x  1 root       root       1.6K Jul 24  2022 get_spanning_reads.R
-rw-rw-r--  1 1389338227 1389338227 3.1G Sep 11  2020 hg38.fa
-rwxrwxr--  1 1389338227 1389338227 125M Jun 19  2015 hg38_genCode22.1.bt2
-rwxrwxr--  1 1389338227 1389338227  71M Jun 19  2015 hg38_genCode22.2.bt2
-rwxrwxr--  1 1389338227 1389338227 1.7M Jun 19  2015 hg38_genCode22.3.bt2
-rwxrwxr--  1 1389338227 1389338227  71M Jun 19  2015 hg38_genCode22.4.bt2
-rw-r--r--  1 1389338227 1389338227  36M Mar  5  2020 hg38_genCode22_blast.nhr
-rw-r--r--  1 1389338227 1389338227 2.3M Mar  5  2020 hg38_genCode22_blast.nin
-rw-r--r--  1 1389338227 1389338227  71M Mar  5  2020 hg38_genCode22_blast.nsq
-rwxrwxr--  1 1389338227 1389338227 308M Jun 19  2015 hg38_genCode22.fa
-rwxrwxr--  1 1389338227 1389338227 125M Jun 19  2015 hg38_genCode22.rev.1.bt2
-rwxrwxr--  1 1389338227 1389338227  71M Jun 19  2015 hg38_genCode22.rev.2.bt2
-rwxrwxr--  1 1389338227 1389338227  41M Jun 19  2015 hg38_genCode22.tab
-rwxrwxr-x  1 root       root       6.5K Jul 24  2022 install_linux64.sh
-rwxrwxr-x  1 root       root       1.2K Jul 24  2022 JAFFA_assembly.groovy
-rwxrwxr-x  1 root       root       1.5K Jul 24  2022 JAFFA_direct.groovy
-rwxrwxr-x  1 root       root       1.7K Jul 24  2022 JAFFA_hybrid.groovy
-rwxrwxr-x  1 root       root       3.0K Jul 24  2022 JAFFAL.groovy
-rw-r--r--  1 root       root       4.9G Apr 25  2021 JAFFA_REFERENCE_FILES_HG38_GENCODE22.V2.tar.gz
-rwxrwxr-x  1 root       root        21K Jul 24  2022 JAFFA_stages.groovy
-rw-r--r--  1 root       root       199K Jul 24  2022 JAFFA-version-2.3.tar.gz
-rw-rw-r--  1 root       root       400K Jul 24  2022 known_fusions.txt
-rw-rw-r--  1 root       root        818 Jul 24  2022 LICENSE
-rwxrwxr-x  1 root       root        18K Jul 24  2022 make_final_table.R
-rwxrwxr--  1 1389338227 1389338227 939M Mar  4  2015 Masked_hg38.1.bt2
-rwxrwxr--  1 1389338227 1389338227 699M Mar  4  2015 Masked_hg38.2.bt2
-rwxrwxr--  1 1389338227 1389338227 2.6M Mar  4  2015 Masked_hg38.3.bt2
-rwxrwxr--  1 1389338227 1389338227 699M Mar  4  2015 Masked_hg38.4.bt2
-rwxrwxr--  1 1389338227 1389338227 939M Mar  4  2015 Masked_hg38.rev.1.bt2
-rwxrwxr--  1 1389338227 1389338227 699M Mar  4  2015 Masked_hg38.rev.2.bt2
-rwxrwxr-x  1 root       root        618 Jul 24  2022 README
-rw-rw-r--  1 root       root        713 Jul 24  2022 README.md
drwxrwxr-x  2 root       root       4.0K Jul 24  2022 scripts
drwxrwxr-x  2 root       root       4.0K Jul 24  2022 src
drwxr-xr-x 13 root       root       4.0K Feb  2 13:14 tools
-rw-r--r--  1 root       root       1017 Feb  2 13:15 tools.groovy
ryleyd@minion:~$ ls -lh /opt/
total 798M
drwxr-xr-x  9 root  root  4.0K Feb 10 16:40 Aeron
-rw-r--r--  1 root  root  738M Oct 18  2022 Anaconda3-2022.10-Linux-x86_64.sh
drwxr-xr-x  9 root  root  4.0K Oct 18  2022 Badread
drwxr-xr-x  3 root  root  4.0K Feb 27 15:20 bsalign
drwx--x--x  4 root  root  4.0K Jul 25  2022 containerd
drwxr-xr-x  4 root  root  4.0K Jan 12 11:18 fusim-0.2.2
drwxr-xr-x  9 root  root  4.0K Feb  3 09:42 genion
drwxr-xr-x  3 root  root  4.0K Jan 16 14:47 google
drwxr-xr-x  3 rhysg users 4.0K Jun 20 14:35 gurobi1002
drwxrwxr-x  3 root  root  4.0K Jun 20 14:12 ibm
drwxrwxr-x  7 root  root  4.0K Feb  2 15:02 JAFFA-version-2.3
drwxr-xr-x  4 root  root  4.0K Jun  8 17:24 nvidia
drwxr-xr-x  8 root  root  4.0K Jun 20 16:11 ont
-rw-r--r--  1 root  root   61M Feb 24  2022 rstudio-server-2022.02.0-443-amd64.deb
-rw-r--r--  1 root  root     0 Mar  2 16:15 test_files02-03-2023test-meta.txt
drwxr-xr-x 12 root  root  4.0K Jun 20 16:34 zoom
ryleyd@minion:~$ ls -lh /data2/ryleyd/benchmark_simulated_reads/positive_control/
total 259G
drwxr-xr-x 15 ryleyd users 4.0K Jun 19 16:06 nanopore
drwxr-xr-x  2 ryleyd users 4.0K Jun  6 13:02 original_fastq
drwxr-xr-x 15 ryleyd users 4.0K Jun 20 14:49 pacbio
-rw-r--r--  1 ryleyd users 6.5G Jun  6 12:23 positive_control_nanopore2020_D100x_M85.fastq.gz
-rw-r--r--  1 ryleyd users  18G Feb 24 22:44 positive_control_nanopore2020_D100x_M90.fastq.gz
-rw-r--r--  1 ryleyd users  46G Feb 25 00:34 positive_control_nanopore2020_D100x_M95.fastq.gz
-rw-r--r--  1 ryleyd users 3.3G May 29 15:40 positive_control_nanopore2020_D10x_M85.fastq.gz
-rw-r--r--  1 ryleyd users 4.7G Feb 24 13:57 positive_control_nanopore2020_D10x_M90.fastq.gz
-rw-r--r--  1 ryleyd users 4.6G Feb 24 14:26 positive_control_nanopore2020_D10x_M95.fastq.gz
-rw-r--r--  1 ryleyd users 3.6G Feb 24 14:54 positive_control_nanopore2020_D20x_M85.fastq.gz
-rw-r--r--  1 ryleyd users 9.3G Feb 24 15:17 positive_control_nanopore2020_D20x_M90.fastq.gz
-rw-r--r--  1 ryleyd users 9.2G Feb 24 16:14 positive_control_nanopore2020_D20x_M95.fastq.gz
-rw-r--r--  1 ryleyd users 4.8G Feb 24 17:10 positive_control_nanopore2020_D50x_M85.fastq.gz
-rw-r--r--  1 ryleyd users  23G Feb 24 17:47 positive_control_nanopore2020_D50x_M90.fastq.gz
-rw-r--r--  1 ryleyd users  23G Feb 24 19:57 positive_control_nanopore2020_D50x_M95.fastq.gz
-rw-r--r--  1 ryleyd users 4.7G Feb 25 12:02 positive_control_pacbio2016_D100x_M85.fastq.gz
-rw-r--r--  1 ryleyd users  11G Feb 25 12:39 positive_control_pacbio2016_D100x_M90.fastq.gz
-rw-r--r--  1 ryleyd users  34G Feb 25 14:13 positive_control_pacbio2016_D100x_M95.fastq.gz
-rw-r--r--  1 ryleyd users 2.5G Feb 25 04:43 positive_control_pacbio2016_D10x_M85.fastq.gz
-rw-r--r--  1 ryleyd users 3.8G Feb 25 05:02 positive_control_pacbio2016_D10x_M90.fastq.gz
-rw-r--r--  1 ryleyd users 3.7G Feb 25 05:31 positive_control_pacbio2016_D10x_M95.fastq.gz
-rw-r--r--  1 ryleyd users 3.0G Feb 25 06:02 positive_control_pacbio2016_D20x_M85.fastq.gz
-rw-r--r--  1 ryleyd users 4.7G Feb 25 06:24 positive_control_pacbio2016_D20x_M90.fastq.gz
-rw-r--r--  1 ryleyd users 7.4G Feb 25 07:02 positive_control_pacbio2016_D20x_M95.fastq.gz
-rw-r--r--  1 ryleyd users 3.7G Feb 25 08:02 positive_control_pacbio2016_D50x_M85.fastq.gz
-rw-r--r--  1 ryleyd users 7.5G Feb 25 08:30 positive_control_pacbio2016_D50x_M90.fastq.gz
-rw-r--r--  1 ryleyd users  19G Feb 25 09:33 positive_control_pacbio2016_D50x_M95.fastq.gz
-rw-r--r--  1 ryleyd users 2.0K May 24 15:21 positive.fastfiles.md5sum.md5
drwxr-xr-x  2 ryleyd users 4.0K Jun 20 12:24 troubleshooting
ryleyd@minion:~$ cd /data2/ryleyd/benchmark_simulated_reads/positive_control/
ryleyd@minion:/data2/ryleyd/benchmark_simulated_reads/positive_control$ ls -lh /data2/ryleyd/benchmark_simulated_reads/positive_control/nanopore/
total 100K
-rw-r--r-- 1 ryleyd users   46 Jun 19 11:51 checks
-rw-r--r-- 1 ryleyd users  43K Jun 20 02:40 commandlog.txt
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 15:57 positive_control_nanopore2020_D100x_M85.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 11:56 positive_control_nanopore2020_D100x_M90.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 20 02:40 positive_control_nanopore2020_D100x_M95.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 15:57 positive_control_nanopore2020_D10x_M85.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 16:08 positive_control_nanopore2020_D10x_M90.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 16:06 positive_control_nanopore2020_D10x_M95.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 13:07 positive_control_nanopore2020_D20x_M85.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 20:15 positive_control_nanopore2020_D20x_M90.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 20:15 positive_control_nanopore2020_D20x_M95.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 16:06 positive_control_nanopore2020_D50x_M85.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 16:13 positive_control_nanopore2020_D50x_M90.fastq
drwxr-xr-x 2 ryleyd users 4.0K Jun 19 23:33 positive_control_nanopore2020_D50x_M95.fastq
RDorney commented 1 year ago

I'll respond later via email with a sample of my dataset that often has problems. I'll also make sure to check if the bases and quality lengths being different for some reads :) Thank you plenty

RDorney commented 1 year ago

with regards to the error about the bases and quality lengths being different for some reads, this actually was the case. Some of the reads simulated by BadRead did not have a quality score line at all! 😮 So far JAFFAL has been the only tool to pick up on this.

nadiadavidson commented 1 year ago

Thanks for the update. Hopefully easy for you to fix the quality lengths in your simulation!

In regards to many of the others errors, it looks like it's failing at the minimap2 stage. Can test whether these commands work outside JAFFAL, for example by running: /opt/JAFFA-version-2.3/tools/bin/minimap2 -t 1 -x splice -c /opt/JAFFA-version-2.3/hg38.fa Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq.fusions.fa > Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf

and/or see if the file Pac_fus_sim_80err.fastq/Pac_fus_sim_80err.fastq_genome.paf etc. are non-empty.

Minimap2 can use a lot of RAM, in particular if you are running several jobs at once, so one thought it that you've hit the memory limit. How much memory does JAFFAL have access too? Do you see the same errors if you run one .fastq file at a time?

Cheers, Nadia.

stefanc-sbg commented 7 months ago

@RDorney I've encountered the same Exit Code : 137 minimap2 issue when I tried running the tool on an AWS instance with 15 GB RAM. However, the error was fixed when running on an instance with more available RAM (I ran it successfully on an instance with 96 GB RAM). @nadiadavidson was right, minimap2 is pretty RAM intensive, running either one sample at a time or running it on a machine with more available RAM should fix this.