search

bioinfo-biols / CIRIquant

circular RNA quantification tools
https://sourceforge.net/projects/ciri/files/CIRIquant
MIT License
27 stars 18 forks source link

Encountered internal HISAT2 exception (#1) #31

Closed cjs0617 closed 3 years ago

cjs0617 commented 3 years ago

Hi Kevin,

I had exactly the same problem as #15 , but I don't really understand what "reducing the number of threads" means. (I only used 4 cores.) Can you elaborate on that? Or it there another way to solve this problem?

Also it's weird. When I was running the test data as a batch file, the run was terminated due to this error. Then I immediately ran the test data in the command line, and I got the desired output in only 2-3 minutes. However, when I ran the experimental data today, running it as a batch file or in the command line both gave this same error. Do you know why does it happen?

Kevinzjy commented 3 years ago

I had the same problem as #15 , but I don't know what "reducing the number of threads" mean. Can you elaborate on that? Or it there another way to solve this problem?

If you are running CIRIquant with a large "-t" option, please reduce this number and try again.

Also it's weird. When I was running the test data as a batch file, the run was terminated due to this error. Then I immediately ran the test data in the command line, and I got the desired output in only 2-3 minutes. However, when I ran the experimental data today, running it as a batch file or in the command line both gave this same error. Do you know why does it happen?

I would speculate that your job monitoring system (e.g. SLURM or Torque) is limiting the total amount of threads or memory for your job, so CIRI2 ran out of resources and didn't finish successfully. Maybe you could require more memory in your batch script and try again.

cjs0617 commented 3 years ago

My -t is 4. Should I reduce that?

Kevinzjy commented 3 years ago

Nope, 4 threads should be fine. Could you post the batch script, log file output/prefix.log and output/CIRIerror.log?

cjs0617 commented 3 years ago

Hi Kevin,

CIRIerror.log is empty.

batch script is:

!/bin/bash

SBATCH -N 1

SBATCH -p RM-shared

SBATCH --ntasks-per-node=8

SBATCH -t 03:00:00

SBATCH --mem-per-cpu=2048

CIRIquant -t 8 \ -1 ./PRJNA234297/SRR1106189/SRR1106189_1.fastq \ -2 ./PRJNA234297/SRR1106189/SRR1106189_2.fastq \ --config ./hg19_test.yml \ -o ./PRJNA234297/SRR1106189 \ -p 189

prefix.log is:

[Thu 2021-06-24 23:30:41] [INFO ] Input reads: SRR1106189_1.fastq,SRR1106189_2.fastq [Thu 2021-06-24 23:30:41] [INFO ] Library type: unstranded [Thu 2021-06-24 23:30:41] [INFO ] Output directory: /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189, Output prefix: 189 [Thu 2021-06-24 23:30:41] [INFO ] Config: hg19 Loaded [Thu 2021-06-24 23:30:41] [INFO ] 256 CPU cores availble, using 8 [Thu 2021-06-24 23:30:41] [INFO ] Align RNA-seq reads to reference genome .. Time loading forward index: 00:00:14 Time loading reference: 00:00:04 Killed (ERR): hisat2-align exited with value 137 [E::sam_parse1] SEQ and QUAL are of different length [W::sam_read1] Parse error at line 43108218 samtools view: error reading file "-" [bam_sort_core] merging from 8 files and 8 in-memory blocks... [Thu 2021-06-24 23:41:16] [INFO ] Estimate gene abundance .. [Thu 2021-06-24 23:43:20] [INFO ] No circRNA information provided, run CIRI2 for junction site prediction .. [Thu 2021-06-24 23:43:20] [INFO ] Running BWA-mem mapping candidate reads .. [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (181, 170430, 120, 173) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (32, 59, 119) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 293) [M::mem_pestat] mean and std.dev: (66.22, 44.60) [M::mem_pestat] low and high boundaries for proper pairs: (1, 380) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.25, 10.93) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (266, 1097, 3819) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 10925) [M::mem_pestat] mean and std.dev: (2305.54, 2685.02) [M::mem_pestat] low and high boundaries for proper pairs: (1, 14478) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (32, 62, 119) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 293) [M::mem_pestat] mean and std.dev: (71.21, 51.28) [M::mem_pestat] low and high boundaries for proper pairs: (1, 380) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@7] performed mate-SW for 84883 reads [M::worker2@6] performed mate-SW for 83002 reads [M::worker2@4] performed mate-SW for 86233 reads [M::worker2@5] performed mate-SW for 84909 reads [M::worker2@1] performed mate-SW for 85040 reads [M::worker2@2] performed mate-SW for 85678 reads [M::worker2@3] performed mate-SW for 85467 reads [M::worker2@0] performed mate-SW for 85348 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (204, 171134, 98, 203) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (37, 68, 117) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 277) [M::mem_pestat] mean and std.dev: (72.33, 47.57) [M::mem_pestat] low and high boundaries for proper pairs: (1, 357) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.25, 10.95) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (207, 1508, 3481) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 10029) [M::mem_pestat] mean and std.dev: (2361.42, 2607.74) [M::mem_pestat] low and high boundaries for proper pairs: (1, 13303) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (36, 78, 130) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 318) [M::mem_pestat] mean and std.dev: (76.36, 53.51) [M::mem_pestat] low and high boundaries for proper pairs: (1, 412) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@0] performed mate-SW for 89048 reads [M::worker2@7] performed mate-SW for 84464 reads [M::worker2@5] performed mate-SW for 86861 reads [M::worker2@2] performed mate-SW for 85339 reads [M::worker2@4] performed mate-SW for 85105 reads [M::worker2@6] performed mate-SW for 89589 reads [M::worker2@3] performed mate-SW for 85518 reads [M::worker2@1] performed mate-SW for 83673 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (219, 170794, 129, 192) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (34, 62, 108) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 256) [M::mem_pestat] mean and std.dev: (65.41, 42.86) [M::mem_pestat] low and high boundaries for proper pairs: (1, 330) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.24, 10.95) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (227, 1258, 4167) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 12047) [M::mem_pestat] mean and std.dev: (2391.39, 2685.81) [M::mem_pestat] low and high boundaries for proper pairs: (1, 15987) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (36, 59, 111) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 261) [M::mem_pestat] mean and std.dev: (66.71, 42.53) [M::mem_pestat] low and high boundaries for proper pairs: (1, 336) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@7] performed mate-SW for 86007 reads [M::worker2@4] performed mate-SW for 86970 reads [M::worker2@6] performed mate-SW for 86320 reads [M::worker2@3] performed mate-SW for 84912 reads [M::worker2@5] performed mate-SW for 88547 reads [M::worker2@2] performed mate-SW for 85014 reads [M::worker2@1] performed mate-SW for 87717 reads [M::worker2@0] performed mate-SW for 88514 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (196, 171220, 120, 187) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (30, 53, 106) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 258) [M::mem_pestat] mean and std.dev: (62.84, 43.30) [M::mem_pestat] low and high boundaries for proper pairs: (1, 334) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.28, 10.95) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (207, 1146, 3649) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 10533) [M::mem_pestat] mean and std.dev: (2218.25, 2701.25) [M::mem_pestat] low and high boundaries for proper pairs: (1, 13975) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (39, 65, 122) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 288) [M::mem_pestat] mean and std.dev: (73.10, 47.31) [M::mem_pestat] low and high boundaries for proper pairs: (1, 371) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@3] performed mate-SW for 85646 reads [M::worker2@6] performed mate-SW for 82218 reads [M::worker2@2] performed mate-SW for 81907 reads [M::worker2@4] performed mate-SW for 81119 reads [M::worker2@1] performed mate-SW for 81943 reads [M::worker2@7] performed mate-SW for 87895 reads [M::worker2@0] performed mate-SW for 82340 reads [M::worker2@5] performed mate-SW for 82461 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (193, 169843, 117, 185) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (37, 69, 132) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 322) [M::mem_pestat] mean and std.dev: (75.10, 49.33) [M::mem_pestat] low and high boundaries for proper pairs: (1, 417) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.26, 11.00) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (179, 1046, 3685) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 10697) [M::mem_pestat] mean and std.dev: (2481.80, 2948.37) [M::mem_pestat] low and high boundaries for proper pairs: (1, 14203) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (34, 66, 125) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 307) [M::mem_pestat] mean and std.dev: (74.64, 54.30) [M::mem_pestat] low and high boundaries for proper pairs: (1, 398) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@6] performed mate-SW for 81920 reads [M::worker2@3] performed mate-SW for 82815 reads [M::worker2@5] performed mate-SW for 84525 reads [M::worker2@2] performed mate-SW for 83992 reads [M::worker2@0] performed mate-SW for 81296 reads [M::worker2@7] performed mate-SW for 84895 reads [M::worker2@1] performed mate-SW for 82998 reads [M::worker2@4] performed mate-SW for 86688 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (197, 169568, 141, 159) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (32, 56, 110) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 266) [M::mem_pestat] mean and std.dev: (65.37, 44.11) [M::mem_pestat] low and high boundaries for proper pairs: (1, 344) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.24, 10.95) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (268, 996, 3949) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 11311) [M::mem_pestat] mean and std.dev: (2483.04, 2783.13) [M::mem_pestat] low and high boundaries for proper pairs: (1, 14992) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (32, 62, 115) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 281) [M::mem_pestat] mean and std.dev: (68.63, 48.84) [M::mem_pestat] low and high boundaries for proper pairs: (1, 364) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@7] performed mate-SW for 81969 reads [M::worker2@4] performed mate-SW for 82311 reads [M::worker2@5] performed mate-SW for 87415 reads [M::worker2@2] performed mate-SW for 87822 reads [M::worker2@3] performed mate-SW for 82453 reads [M::worker2@6] performed mate-SW for 86192 reads [M::worker2@1] performed mate-SW for 83764 reads [M::worker2@0] performed mate-SW for 83861 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (200, 170740, 113, 193) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (32, 64, 124) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 308) [M::mem_pestat] mean and std.dev: (72.45, 54.87) [M::mem_pestat] low and high boundaries for proper pairs: (1, 400) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.24, 10.97) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (318, 850, 2894) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 8046) [M::mem_pestat] mean and std.dev: (1584.33, 1911.83) [M::mem_pestat] low and high boundaries for proper pairs: (1, 10622) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (31, 59, 116) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 286) [M::mem_pestat] mean and std.dev: (66.46, 45.58) [M::mem_pestat] low and high boundaries for proper pairs: (1, 371) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@2] performed mate-SW for 83061 reads [M::worker2@4] performed mate-SW for 82057 reads [M::worker2@3] performed mate-SW for 81531 reads [M::worker2@6] performed mate-SW for 84302 reads [M::worker2@7] performed mate-SW for 83672 reads [M::worker2@0] performed mate-SW for 83406 reads [M::worker2@1] performed mate-SW for 83338 reads [M::worker2@5] performed mate-SW for 82322 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (206, 171668, 133, 174) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (30, 59, 119) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 297) [M::mem_pestat] mean and std.dev: (66.44, 45.89) [M::mem_pestat] low and high boundaries for proper pairs: (1, 386) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.16, 10.99) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (282, 1659, 3281) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 9279) [M::mem_pestat] mean and std.dev: (2088.02, 2123.93) [M::mem_pestat] low and high boundaries for proper pairs: (1, 12278) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (40, 74, 124) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 292) [M::mem_pestat] mean and std.dev: (72.03, 42.45) [M::mem_pestat] low and high boundaries for proper pairs: (1, 376) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@6] performed mate-SW for 84291 reads [M::worker2@1] performed mate-SW for 85331 reads [M::worker2@7] performed mate-SW for 82575 reads [M::worker2@2] performed mate-SW for 80905 reads [M::worker2@3] performed mate-SW for 83294 reads [M::worker2@5] performed mate-SW for 83931 reads [M::worker2@0] performed mate-SW for 84522 reads [M::worker2@4] performed mate-SW for 85611 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (216, 171209, 139, 201) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (33, 72, 123) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 303) [M::mem_pestat] mean and std.dev: (76.20, 52.06) [M::mem_pestat] low and high boundaries for proper pairs: (1, 393) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.26, 10.97) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (171, 859, 3231) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 9351) [M::mem_pestat] mean and std.dev: (1953.91, 2396.25) [M::mem_pestat] low and high boundaries for proper pairs: (1, 12411) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (34, 67, 117) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 283) [M::mem_pestat] mean and std.dev: (70.30, 46.95) [M::mem_pestat] low and high boundaries for proper pairs: (1, 366) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@2] performed mate-SW for 86228 reads [M::worker2@7] performed mate-SW for 83517 reads [M::worker2@5] performed mate-SW for 82460 reads [M::worker2@4] performed mate-SW for 82255 reads [M::worker2@6] performed mate-SW for 83230 reads [M::worker2@1] performed mate-SW for 82744 reads [M::worker2@0] performed mate-SW for 81556 reads [M::worker2@3] performed mate-SW for 82959 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (158, 171552, 121, 172) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (39, 73, 123) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 291) [M::mem_pestat] mean and std.dev: (78.09, 54.33) [M::mem_pestat] low and high boundaries for proper pairs: (1, 375) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.22, 10.95) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (283, 1333, 4439) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 12751) [M::mem_pestat] mean and std.dev: (2513.50, 2652.22) [M::mem_pestat] low and high boundaries for proper pairs: (1, 16907) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (35, 72, 121) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 293) [M::mem_pestat] mean and std.dev: (72.46, 48.28) [M::mem_pestat] low and high boundaries for proper pairs: (1, 379) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@4] performed mate-SW for 85320 reads [M::worker2@6] performed mate-SW for 86449 reads [M::worker2@2] performed mate-SW for 86417 reads [M::worker2@7] performed mate-SW for 88486 reads [M::worker2@3] performed mate-SW for 86492 reads [M::worker2@1] performed mate-SW for 87923 reads [M::worker2@5] performed mate-SW for 88189 reads [M::worker2@0] performed mate-SW for 84551 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (201, 170470, 123, 187) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (28, 52, 107) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 265) [M::mem_pestat] mean and std.dev: (62.19, 42.97) [M::mem_pestat] low and high boundaries for proper pairs: (1, 344) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.21, 10.97) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (384, 1928, 4433) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 12531) [M::mem_pestat] mean and std.dev: (2840.59, 2906.14) [M::mem_pestat] low and high boundaries for proper pairs: (1, 16580) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (34, 69, 116) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 280) [M::mem_pestat] mean and std.dev: (73.45, 48.89) [M::mem_pestat] low and high boundaries for proper pairs: (1, 362) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@4] performed mate-SW for 90573 reads [M::worker2@5] performed mate-SW for 84990 reads [M::worker2@7] performed mate-SW for 87735 reads [M::worker2@6] performed mate-SW for 88823 reads [M::worker2@2] performed mate-SW for 88769 reads [M::worker2@0] performed mate-SW for 89911 reads [M::worker2@1] performed mate-SW for 87861 reads [M::worker2@3] performed mate-SW for 90391 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (228, 169752, 106, 182) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (30, 58, 119) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 297) [M::mem_pestat] mean and std.dev: (70.60, 53.07) [M::mem_pestat] low and high boundaries for proper pairs: (1, 386) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.27, 10.92) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (277, 948, 3113) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 8785) [M::mem_pestat] mean and std.dev: (1872.71, 2141.08) [M::mem_pestat] low and high boundaries for proper pairs: (1, 11621) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (33, 56, 110) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 264) [M::mem_pestat] mean and std.dev: (65.39, 41.72) [M::mem_pestat] low and high boundaries for proper pairs: (1, 341) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@0] performed mate-SW for 91085 reads [M::worker2@4] performed mate-SW for 90799 reads [M::worker2@7] performed mate-SW for 94376 reads [M::worker2@6] performed mate-SW for 93807 reads [M::worker2@5] performed mate-SW for 90616 reads [M::worker2@2] performed mate-SW for 90481 reads [M::worker2@1] performed mate-SW for 90540 reads [M::worker2@3] performed mate-SW for 91104 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (214, 170396, 118, 182) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (35, 63, 128) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 314) [M::mem_pestat] mean and std.dev: (69.85, 51.07) [M::mem_pestat] low and high boundaries for proper pairs: (1, 407) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.26, 10.94) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (393, 971, 3010) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 8244) [M::mem_pestat] mean and std.dev: (1868.09, 2134.02) [M::mem_pestat] low and high boundaries for proper pairs: (1, 10861) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (33, 58, 111) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 267) [M::mem_pestat] mean and std.dev: (67.29, 43.16) [M::mem_pestat] low and high boundaries for proper pairs: (1, 345) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@7] performed mate-SW for 90268 reads [M::worker2@2] performed mate-SW for 87676 reads [M::worker2@6] performed mate-SW for 92192 reads [M::worker2@4] performed mate-SW for 89092 reads [M::worker2@3] performed mate-SW for 95333 reads [M::worker2@5] performed mate-SW for 90610 reads [M::worker2@1] performed mate-SW for 90578 reads [M::worker2@0] performed mate-SW for 90142 reads [M::main_mem] read 888890 sequences (80000100 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (199, 169925, 109, 172) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (30, 55, 106) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 258) [M::mem_pestat] mean and std.dev: (62.86, 44.52) [M::mem_pestat] low and high boundaries for proper pairs: (1, 334) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (180, 188, 196) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (148, 228) [M::mem_pestat] mean and std.dev: (186.20, 10.94) [M::mem_pestat] low and high boundaries for proper pairs: (132, 244) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (149, 1014, 2784) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 8054) [M::mem_pestat] mean and std.dev: (1570.54, 1845.13) [M::mem_pestat] low and high boundaries for proper pairs: (1, 10689) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (38, 78, 133) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 323) [M::mem_pestat] mean and std.dev: (73.58, 49.55) [M::mem_pestat] low and high boundaries for proper pairs: (1, 418) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::worker2@4] performed mate-SW for 86822 reads [M::worker2@5] performed mate-SW for 88445 reads [M::worker2@2] performed mate-SW for 86894 reads [M::worker2@6] performed mate-SW for 90756 reads [M::worker2@7] performed mate-SW for 85596 reads [M::worker2@1] performed mate-SW for 86320 reads [M::worker2@0] performed mate-SW for 87648 reads [M::worker2@3] performed mate-SW for 90065 reads [M::main_mem] read 888890 sequences (80000100 bp)... /opt/packages/BWA/0.7.3a/bwa: line 21: 281895 Killed singularity exec $OPTIONS $DIRECTORY/singularity-$PACKAGE-$VERSION.sif $TOOL "$@" [Thu 2021-06-24 23:52:43] [INFO ] Running CIRI2 for circRNA detection .. [Thu Jun 24 23:52:43 2021] CIRI begins running [Thu Jun 24 23:52:43 2021] Loading reference Output file /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189.ciri already exists and is overwritten. [Thu Jun 24 23:53:08 2021] Requesting system to split SAM into 8 pieces [Thu Jun 24 23:52:43 2021] CIRI begins running [Thu Jun 24 23:52:43 2021] Loading reference [Thu Jun 24 23:53:08 2021] Requesting system to split SAM into 8 pieces Divided SAM sizes: /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_unmapped.samaa 460324864 /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_unmapped.samab 460324864 /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_unmapped.samac 460324864 /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_unmapped.samad 460324864 /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_unmapped.samae 460324864 /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_unmapped.samaf 460324864 /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_unmapped.samag 460324864 /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_unmapped.samah 460324864 SAM was divided successfully. First read of divided SAM files: 189_unmapped.samab: SRR1106189.780729 189_unmapped.samac: SRR1106189.1559207 189_unmapped.samad: SRR1106189.2335080 189_unmapped.samae: SRR1106189.3114744 189_unmapped.samaf: SRR1106189.3895575 189_unmapped.samag: SRR1106189.4675404 189_unmapped.samah: SRR1106189.5446785 189_unmapped.samaa: SRR1106189.1 First reads were recorded successfully. [Thu Jun 24 23:53:50 2021] First scanning [Thu Jun 24 23:53:50 2021] First scanning Worker 1 begins to scan 189_unmapped.samaf. Worker 2 begins to scan 189_unmapped.samah. Worker 3 begins to scan 189_unmapped.samad. Worker 4 begins to scan 189_unmapped.samae. Worker 5 begins to scan 189_unmapped.samaa. Worker 6 begins to scan 189_unmapped.samab. Worker 7 begins to scan 189_unmapped.samac. [Thu 2021-06-24 23:54:14] [INFO ] Extract circular sequence [Thu 2021-06-24 23:54:14] [INFO ] Building circular index .. Settings: Output files: "/ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_index..ht2" Line rate: 6 (line is 64 bytes) Lines per side: 1 (side is 64 bytes) Offset rate: 4 (one in 16) FTable chars: 10 Strings: unpacked Local offset rate: 3 (one in 8) Local fTable chars: 6 Local sequence length: 57344 Local sequence overlap between two consecutive indexes: 1024 Endianness: little Actual local endianness: little Sanity checking: disabled Assertions: disabled Random seed: 0 Sizeofs: void:8, int:4, long:8, size_t:8 Input files DNA, FASTA: /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_index.fa Warning: Empty fasta file: '/ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_index.fa' Warning: All fasta inputs were empty Total time for call to driver() for forward index: 00:00:00 Error: Encountered internal HISAT2 exception (#1) Command: hisat2-build --wrapper basic-0 -p 8 -f /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_index.fa /ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_index [Thu 2021-06-24 23:54:14] [INFO ] De novo alignment for circular RNAs .. (ERR): "/ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_index" does not exist Exiting now ... [main_samview] fail to read the header from "-". samtools sort: failed to read header from "/ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_denovo.bam" [E::hts_open_format] Failed to open file "/ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_denovo.sorted.bam" : No such file or directory samtools index: failed to open "/ocean/projects/mcb200049p/jshi8/PRJNA234297/SRR1106189/circ/189_denovo.sorted.bam": No such file or directory

Kevinzjy commented 3 years ago

[Thu 2021-06-24 23:30:41] [INFO ] Align RNA-seq reads to reference genome .. Time loading forward index: 00:00:14 Time loading reference: 00:00:04 Killed (ERR): hisat2-align exited with value 137

This "Killed" error means your script has exceeded the maximum memory allocated. Maybe you could change the memory specified at the header lines of your batch script from #SBATCH --mem-per-cpu=2048 to #SBATCH --mem=65535.

cjs0617 commented 3 years ago

It says: sbatch: error: Allocation requested mem-per-core higher than maximum of 2000M/core sbatch: error: Batch job submission failed: Access/permission denied

Kevinzjy commented 3 years ago

Well, then you have to specify more cores in your batch script. It seems that your server administrator has limited the maximum memory per core.

cjs0617 commented 3 years ago

Hi Kevin,

We solved this problem by deleting useless files as well as increasing tasks-per-node and threads. Thanks for you help.