search

bioinfo-biols / CIRIquant

circular RNA quantification tools
https://sourceforge.net/projects/ciri/files/CIRIquant
MIT License
27 stars 17 forks source link

.fa is empty, index does not exist #35

Open algaebrown opened 2 years ago

algaebrown commented 2 years ago

I had an error with CIRIquant. starting from Warning: Empty fasta file: '/oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_index.fa' then (ERR): "/oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_index" does not exist. I ran with 16 threads. CIRIerror.log is empty. Please help. Thanks

Full log:

[Fri 2022-05-27 17:20:13] [INFO ] Running CIRI2 for circRNA detection ..
[Fri May 27 17:20:14 2022] CIRI begins running
[Fri May 27 17:20:14 2022] Loading reference
[Fri May 27 17:20:47 2022] Requesting system to split SAM into 16 pieces
[Fri May 27 17:20:14 2022] CIRI begins running
[Fri May 27 17:20:14 2022] Loading reference
[Fri May 27 17:20:47 2022] Requesting system to split SAM into 16 pieces
 Divided SAM sizes:
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samaa         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samab         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samac         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samad         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samae         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samaf         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samag         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samah         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samai         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samaj         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samak         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samal         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samam         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.saman         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samao         4395350130
 /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_unmapped.samap         4395350121
 SAM was divided successfully.
 First read of divided SAM files: 
 APO-1-R_unmapped.samab: A00475:448:HFLNKDRX2:1:2121:1081:20055
 APO-1-R_unmapped.samac: A00475:448:HFLNKDRX2:1:2141:15637:24533
 APO-1-R_unmapped.samad: A00475:448:HFLNKDRX2:1:2161:14235:34507
 APO-1-R_unmapped.samae: A00475:448:HFLNKDRX2:1:2204:3106:5431
 APO-1-R_unmapped.samaf: A00475:448:HFLNKDRX2:1:2223:16224:31125
 APO-1-R_unmapped.samag: A00475:448:HFLNKDRX2:1:2243:20509:16626
 APO-1-R_unmapped.samah: A00475:448:HFLNKDRX2:1:2262:8730:32424
 APO-1-R_unmapped.samai: A00475:448:HFLNKDRX2:2:2104:7012:25848
 APO-1-R_unmapped.samaj: A00475:448:HFLNKDRX2:2:2125:16559:12696
 APO-1-R_unmapped.samak: A00475:448:HFLNKDRX2:2:2145:27679:10316
 APO-1-R_unmapped.samal: A00475:448:HFLNKDRX2:2:2163:20998:27743
 APO-1-R_unmapped.samam: A00475:448:HFLNKDRX2:2:2204:4562:18067
 APO-1-R_unmapped.saman: A00475:448:HFLNKDRX2:2:2223:21305:14779
 APO-1-R_unmapped.samao: A00475:448:HFLNKDRX2:2:2242:14931:8782
 APO-1-R_unmapped.samap: A00475:448:HFLNKDRX2:2:2260:7943:23265
 APO-1-R_unmapped.samaa: A00475:448:HFLNKDRX2:1:2101:1018:1000
 First reads were recorded successfully.
[Fri May 27 17:39:25 2022] First scanning
[Fri May 27 17:39:25 2022] First scanning
 Worker 1 begins to scan APO-1-R_unmapped.samal.
 Worker 2 begins to scan APO-1-R_unmapped.samaj.
 Worker 3 begins to scan APO-1-R_unmapped.samam.
 Worker 4 begins to scan APO-1-R_unmapped.samad.
 Worker 5 begins to scan APO-1-R_unmapped.samag.
 Worker 6 begins to scan APO-1-R_unmapped.samaf.
 Worker 7 begins to scan APO-1-R_unmapped.samai.
 Worker 8 begins to scan APO-1-R_unmapped.samac.
 Worker 9 begins to scan APO-1-R_unmapped.samah.
 Worker 10 begins to scan APO-1-R_unmapped.samab.
 Worker 11 begins to scan APO-1-R_unmapped.samak.
 Worker 12 begins to scan APO-1-R_unmapped.samae.
[Fri 2022-05-27 17:41:16] [INFO ] Extract circular sequence
[Fri 2022-05-27 17:41:16] [INFO ] Building circular index ..
Settings:
  Output files: "/oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_index.*.ht2"
  Line rate: 6 (line is 64 bytes)
  Lines per side: 1 (side is 64 bytes)
  Offset rate: 4 (one in 16)
  FTable chars: 10
  Strings: unpacked
  Local offset rate: 3 (one in 8)
  Local fTable chars: 6
  Local sequence length: 57344
  Local sequence overlap between two consecutive indexes: 1024
  Endianness: little
  Actual local endianness: little
  Sanity checking: disabled
  Assertions: disabled
  Random seed: 0
  Sizeofs: void*:8, int:4, long:8, size_t:8
Input files DNA, FASTA:
  /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_index.fa
Warning: Empty fasta file: '/oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_index.fa'
Warning: All fasta inputs were empty
Total time for call to driver() for forward index: 00:00:00
Error: Encountered internal HISAT2 exception (#1)
Command: hisat2-build --wrapper basic-0 -p 16 -f /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_index.fa /oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_index 
[Fri 2022-05-27 17:41:20] [INFO ] De novo alignment for circular RNAs ..
(ERR): "/oasis/tscc/scratch/hsher/circSTAMP_pipe/output/circ/APO-1-R_index" does not exist
Exiting now ...
Kevinzjy commented 2 years ago

Hi @algaebrown, it seems that you have probably run out of memory when running CIRI2. Please reduce the thread number and try again (similar to #31).

algaebrown commented 6 months ago

Hi @Kevinzjy, how much memory approximately do you need per thread? I am trying to make it run faster with more threads.

Or is it possible to run it stage by stage? For example, Run until RNAseq alignment for 1 step, save the files. use the files to run denovo...etc. Cause it kept dying in the middle and had to start from scratch again, which is annoying.

sometimes we have a very large dataset. Is there a workaround for it?

Any comment will be helpful. Thanks!!

Kevinzjy commented 6 months ago

Hi @algaebrown , that depends on the size of your dataset.

You could first run BWA and CIRI2 manually to get the circRNA list (https://ciri-cookbook.readthedocs.io/en/latest/CIRI2.html#an-example-of-running-ciri2).

Then run CIRIquant using the output from CIRI2 (refer to https://ciri-cookbook.readthedocs.io/en/latest/CIRIquant_2_quantification.html#quantify-circrnas-using-results-from-other-tools)

CIRIquant -t 4 \
          -1 ./test_1.fq.gz \
          -2 ./test_2.fq.gz \
          --config ./chr1.yml \
          -o ./test \
          -p test \
          --circ CIRI2_output.list \
          --tool CIRI2
algaebrown commented 6 months ago

Thank you! I will try.

Charlene On Dec 22, 2023, 10:46 PM -0800, Jinyang Zhang @.***>, wrote:

Hi @algaebrown , that depends on the size of your dataset. You could first run BWA and CIRI2 manually to get the circRNA list (https://ciri-cookbook.readthedocs.io/en/latest/CIRI2.html#an-example-of-running-ciri2). Then run CIRIquant using the output from CIRI2 (refer to https://ciri-cookbook.readthedocs.io/en/latest/CIRIquant_2_quantification.html#quantify-circrnas-using-results-from-other-tools) CIRIquant -t 4 \ -1 ./test_1.fq.gz \ -2 ./test_2.fq.gz \ --config ./chr1.yml \ -o ./test \ -p test \ --circ CIRI2_output.list \ --tool CIRI2 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>