search

algaebrown / circSTAMP_pipe

0 stars 0 forks source link

Error in rule ciri_run_ciri_RNASE: #2

Closed byee4 closed 2 months ago

byee4 commented 2 months ago

I assume this is some kind of latency issue, but just in case you've hit the same issue and it's a reproducible one. I'll close it if I can re-trying fixes things.

Error in rule ciri_run_ciri_RNASE:
    jobid: 56
    output: output/align/RBM15_2.sorted.bam, output/align/RBM15_2.sorted.bam.bai, output/RBM15_2.gtf, output/RBM15_2.bed, output/gene/RBM15_2_cov.gtf, output/gene/RBM15_2_genes.list, output/gene/RBM15_2_out.gtf, output/circ/RBM15_2.ciri, output/circ/RBM15_2_index.fa, output/circ/RBM15_2_denovo.sorted.bam, output/circ/RBM15_2_denovo.sorted.bam.bai
    shell:

        CIRIquant             -1 output/fastqs/RBM15_2-trimmed-pair1.fastq.gz             -2 output/fastqs/RBM15_2-trimmed-pair2.fastq.gz             --config /tscc/projects/ps-yeolab4/software/circstamp/a7c4316/examples/ciriconfig_full_mm10.yaml             --library-type 2             -o output/             -p RBM15_2             -t 16

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
    cluster_jobid: Submitted batch job 2210694

...

Error executing rule ciri_run_ciri_RNASE on cluster (jobid: 56, external: Submitted batch job 2210694, jobscript: /tscc/lustre/ddn/scratch/bay001/tao_circrna_20240812/temporary_data/RBM15_OE_KD_rar11_N2A/.snakemake/tmp.os93625t/snakejob.ciri_run_ciri_RNASE.56.sh). For error details see the cluster log and the log files of the involved rule(s).

Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 64
Rules claiming more threads will be scaled down.
Select jobs to execute...

[Tue Aug 13 08:49:29 2024]
rule ciri_run_ciri_RNASE:
    input: /tscc/projects/ps-yeolab3/bay001/annotations/mm10/mm10.fa.amb, /tscc/projects/ps-yeolab3/bay001/annotations/mm10/hisat2_vM25/mm10.1.ht2, output/fastqs/RBM15_2-trimmed-pair1.fastq.gz, output/fastqs/RBM15_2-trimmed-pair2.fastq.gz, /tscc/projects/ps-yeolab4/software/circstamp/a7c4316/examples/ciriconfig_full_mm10.yaml
    output: output/align/RBM15_2.sorted.bam, output/align/RBM15_2.sorted.bam.bai, output/RBM15_2.gtf, output/RBM15_2.bed, output/gene/RBM15_2_cov.gtf, output/gene/RBM15_2_genes.list, output/gene/RBM15_2_out.gtf, output/circ/RBM15_2.ciri, output/circ/RBM15_2_index.fa, output/circ/RBM15_2_denovo.sorted.bam, output/circ/RBM15_2_denovo.sorted.bam.bai
    jobid: 0
    benchmark: benchmarks/ciriquant.RBM15_2.txt
    wildcards: sample_label=RBM15_2
    resources: mem_mb=16000, disk_mb=11912, tmpdir=, cpus=1, runtime=60, nodes=1, partition=condo

        CIRIquant             -1 output/fastqs/RBM15_2-trimmed-pair1.fastq.gz             -2 output/fastqs/RBM15_2-trimmed-pair2.fastq.gz             --config /tscc/projects/ps-yeolab4/software/circstamp/a7c4316/examples/ciriconfig_full_mm10.yaml             --library-type 2             -o output/             -p RBM15_2             -t 16

Activating singularity image /tscc/projects/ps-yeolab4/software/circstamp/c89ea98/bin/.singularity/17615172ccba48cb6ed25b00d5d08e49.simg
[Tue 2024-08-13 08:49:30] [INFO ] Input reads: RBM15_2-trimmed-pair1.fastq.gz,RBM15_2-trimmed-pair2.fastq.gz
[Tue 2024-08-13 08:49:30] [INFO ] Library type: TAKARA SMARTer
[Tue 2024-08-13 08:49:30] [INFO ] Output directory: /tscc/lustre/ddn/scratch/bay001/tao_circrna_20240812/temporary_data/RBM15_OE_KD_rar11_N2A/output, Output prefix: RBM15_2
[Tue 2024-08-13 08:49:30] [INFO ] Config: mm10_gencode_vM25 Loaded
[Tue 2024-08-13 08:49:30] [INFO ] 64 CPU cores availble, using 16
[Tue 2024-08-13 08:49:30] [INFO ] Align RNA-seq reads to reference genome ..
[Tue 2024-08-13 09:01:16] [INFO ] Estimate gene abundance ..
[Tue 2024-08-13 09:04:29] [INFO ] No circRNA information provided, run CIRI2 for junction site prediction ..
[Tue 2024-08-13 09:04:29] [INFO ] Running BWA-mem mapping candidate reads ..
[Tue 2024-08-13 09:43:46] [INFO ] Running CIRI2 for circRNA detection ..
Traceback (most recent call last):
  File "/usr/local/bin/CIRIquant", line 11, in <module>
    load_entry_point('CIRIquant==1.1.2', 'console_scripts', 'CIRIquant')()
  File "/usr/local/lib/python2.7/site-packages/CIRIquant-1.1.2-py2.7.egg/CIRIquant/main.py", line 180, in main
    circ_parser.convert(bed_file)
  File "/usr/local/lib/python2.7/site-packages/CIRIquant-1.1.2-py2.7.egg/CIRIquant/utils.py", line 239, in convert
    circ_data = getattr(self, '_' + self.tool.lower())()
  File "/usr/local/lib/python2.7/site-packages/CIRIquant-1.1.2-py2.7.egg/CIRIquant/utils.py", line 146, in _ciri2
    with open(self.circ, 'r') as f:
IOError: [Errno 2] No such file or directory: '/tscc/lustre/ddn/scratch/bay001/tao_circrna_20240812/temporary_data/RBM15_OE_KD_rar11_N2A/output/circ/RBM15_2.ciri'
[Tue Aug 13 09:44:36 2024]
Error in rule ciri_run_ciri_RNASE:
    jobid: 0
    output: output/align/RBM15_2.sorted.bam, output/align/RBM15_2.sorted.bam.bai, output/RBM15_2.gtf, output/RBM15_2.bed, output/gene/RBM15_2_cov.gtf, output/gene/RBM15_2_genes.list, output/gene/RBM15_2_out.gtf, output/circ/RBM15_2.ciri, output/circ/RBM15_2_index.fa, output/circ/RBM15_2_denovo.sorted.bam, output/circ/RBM15_2_denovo.sorted.bam.bai
    shell:

        CIRIquant             -1 output/fastqs/RBM15_2-trimmed-pair1.fastq.gz             -2 output/fastqs/RBM15_2-trimmed-pair2.fastq.gz             --config /tscc/projects/ps-yeolab4/software/circstamp/a7c4316/examples/ciriconfig_full_mm10.yaml             --library-type 2             -o output/             -p RBM15_2             -t 16

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job ciri_run_ciri_RNASE since they might be corrupted:
output/align/RBM15_2.sorted.bam, output/align/RBM15_2.sorted.bam.bai, output/gene/RBM15_2_cov.gtf, output/gene/RBM15_2_genes.list, output/gene/RBM15_2_out.gtf
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
byee4 commented 2 months ago

Seems to be non-reproducible, so I'll close this issue.

Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 64
Rules claiming more threads will be scaled down.
Select jobs to execute...

[Tue Aug 13 13:18:12 2024]
rule ciri_run_ciri_RNASE:
    input: /tscc/projects/ps-yeolab3/bay001/annotations/mm10/mm10.fa.amb, /tscc/projects/ps-yeolab3/bay001/annotations/mm10/hisat2_vM25/mm10.1.ht2, output/fastqs/RBM15_2-trimmed-pair1.fastq.gz, output/fastqs/RBM15_2-trimmed-pair2.fastq.gz, /tscc/projects/ps-yeolab4/software/circstamp/a7c4316/examples/ciriconfig_full_mm10.yaml
    output: output/align/RBM15_2.sorted.bam, output/align/RBM15_2.sorted.bam.bai, output/RBM15_2.gtf, output/RBM15_2.bed, output/gene/RBM15_2_cov.gtf, output/gene/RBM15_2_genes.list, output/gene/RBM15_2_out.gtf, output/circ/RBM15_2.ciri, output/circ/RBM15_2_index.fa, output/circ/RBM15_2_denovo.sorted.bam, output/circ/RBM15_2_denovo.sorted.bam.bai
    jobid: 0
    benchmark: benchmarks/ciriquant.RBM15_2.txt
    wildcards: sample_label=RBM15_2
    resources: mem_mb=16000, disk_mb=11912, tmpdir=, cpus=1, runtime=60, nodes=1, partition=condo

        CIRIquant             -1 output/fastqs/RBM15_2-trimmed-pair1.fastq.gz             -2 output/fastqs/RBM15_2-trimmed-pair2.fastq.gz             --config /tscc/projects/ps-yeolab4/software/circstamp/a7c4316/examples/ciriconfig_full_mm10.yaml             --library-type 2             -o output/             -p RBM15_2             -t 16

Activating singularity image /tscc/projects/ps-yeolab4/software/circstamp/c89ea98/bin/.singularity/17615172ccba48cb6ed25b00d5d08e49.simg
[Tue 2024-08-13 13:18:13] [INFO ] Input reads: RBM15_2-trimmed-pair1.fastq.gz,RBM15_2-trimmed-pair2.fastq.gz
[Tue 2024-08-13 13:18:13] [INFO ] Library type: TAKARA SMARTer
[Tue 2024-08-13 13:18:13] [INFO ] Output directory: /tscc/lustre/ddn/scratch/bay001/tao_circrna_20240812/temporary_data/RBM15_OE_KD_rar11_N2A/output, Output prefix: RBM15_2
[Tue 2024-08-13 13:18:13] [INFO ] Config: mm10_gencode_vM25 Loaded
[Tue 2024-08-13 13:18:13] [INFO ] 64 CPU cores availble, using 16
[Tue 2024-08-13 13:18:13] [INFO ] Align RNA-seq reads to reference genome ..
[Tue 2024-08-13 13:29:57] [INFO ] Estimate gene abundance ..
[Tue 2024-08-13 13:33:05] [INFO ] No circRNA information provided, run CIRI2 for junction site prediction ..
[Tue 2024-08-13 13:33:05] [INFO ] Running BWA-mem mapping candidate reads ..
[Tue 2024-08-13 14:12:21] [INFO ] Running CIRI2 for circRNA detection ..
[Tue 2024-08-13 14:22:46] [INFO ] Extract circular sequence

[Tue 2024-08-13 14:22:46] [0%   ] [..................................................]
[Tue 2024-08-13 14:22:46] [4%   ] [##................................................]
[Tue 2024-08-13 14:22:47] [8%   ] [####..............................................]
[Tue 2024-08-13 14:22:47] [12%  ] [######............................................]
[Tue 2024-08-13 14:22:48] [16%  ] [########..........................................]
[Tue 2024-08-13 14:22:48] [20%  ] [##########........................................]
[Tue 2024-08-13 14:22:49] [24%  ] [############......................................]
[Tue 2024-08-13 14:22:49] [28%  ] [##############....................................]
[Tue 2024-08-13 14:22:49] [32%  ] [################..................................]
[Tue 2024-08-13 14:22:50] [36%  ] [##################................................]
[Tue 2024-08-13 14:22:50] [40%  ] [####################..............................]
[Tue 2024-08-13 14:22:50] [44%  ] [######################............................]
[Tue 2024-08-13 14:22:51] [48%  ] [########################..........................]
[Tue 2024-08-13 14:22:51] [52%  ] [##########################........................]
[Tue 2024-08-13 14:22:52] [56%  ] [############################......................]
[Tue 2024-08-13 14:22:52] [60%  ] [##############################....................]
[Tue 2024-08-13 14:22:52] [64%  ] [################################..................]
[Tue 2024-08-13 14:22:53] [68%  ] [##################################................]
[Tue 2024-08-13 14:22:53] [72%  ] [####################################..............]
[Tue 2024-08-13 14:22:53] [76%  ] [######################################............]
[Tue 2024-08-13 14:22:54] [80%  ] [########################################..........]
[Tue 2024-08-13 14:22:54] [84%  ] [##########################################........]
[Tue 2024-08-13 14:22:54] [88%  ] [############################################......]
[Tue 2024-08-13 14:22:54] [92%  ] [##############################################....]
[Tue 2024-08-13 14:22:54] [96%  ] [################################################..]
[Tue 2024-08-13 14:22:54] [100% ] [##################################################]
[Tue 2024-08-13 14:22:54] [INFO ] Building circular index ..
[Tue 2024-08-13 14:45:20] [INFO ] De novo alignment for circular RNAs ..
[Tue 2024-08-13 14:58:36] [INFO ] Detecting reads containing Back-splicing signals
[Tue 2024-08-13 15:01:02] [INFO ] Detecting FSJ reads from genome alignment file
[Tue 2024-08-13 15:09:17] [INFO ] Merge bsj and fsj results
[Tue 2024-08-13 15:09:17] [INFO ] Loading annotation gtf ..
[Tue 2024-08-13 15:09:23] [INFO ] Output circRNA expression values
[Tue 2024-08-13 15:09:55] [WARNING] chrom of contig "chr4_GL456216_random" not in annotation gtf, please check
[Tue 2024-08-13 15:09:55] [WARNING] chrom of contig "chrUn_GL456367" not in annotation gtf, please check
[Tue 2024-08-13 15:09:55] [WARNING] chrom of contig "chrUn_GL456379" not in annotation gtf, please check
[Tue 2024-08-13 15:09:55] [WARNING] chrom of contig "chrUn_JH584304" not in annotation gtf, please check
[Tue 2024-08-13 15:09:55] [WARNING] chrom of contig "chrUn_JH584304" not in annotation gtf, please check
[Tue 2024-08-13 15:09:55] [WARNING] chrom of contig "chrUn_JH584304" not in annotation gtf, please check
[Tue 2024-08-13 15:09:55] [WARNING] chrom of contig "chrUn_JH584304" not in annotation gtf, please check
[Tue 2024-08-13 15:09:55] [WARNING] chrom of contig "chrUn_JH584304" not in annotation gtf, please check
[Tue 2024-08-13 15:09:55] [WARNING] chrom of contig "chrUn_JH584304" not in annotation gtf, please check
[Tue 2024-08-13 15:09:55] [WARNING] chrom of contig "chrUn_JH584304" not in annotation gtf, please check
[Tue 2024-08-13 15:09:55] [WARNING] chrom of contig "chrUn_JH584304" not in annotation gtf, please check
[Tue 2024-08-13 15:09:56] [WARNING] chrom of contig "chrX_GL456233_random" not in annotation gtf, please check
[Tue 2024-08-13 15:09:56] [WARNING] chrom of contig "chrX_GL456233_random" not in annotation gtf, please check
[Tue 2024-08-13 15:09:56] [WARNING] chrom of contig "chrX_GL456233_random" not in annotation gtf, please check
[Tue 2024-08-13 15:09:56] [WARNING] chrom of contig "chrX_GL456233_random" not in annotation gtf, please check
[Tue 2024-08-13 15:10:05] [INFO ] circRNA Expression profile: RBM15_2.gtf
[Tue 2024-08-13 15:10:05] [INFO ] Finished!
[Tue Aug 13 15:10:05 2024]
Finished job 0.
1 of 1 steps (100%) done