liulab-dfci / RIMA_pipeline

70 stars 21 forks source link

Job killed before alignment is finished #14

Open leeanapeters opened 2 years ago

leeanapeters commented 2 years ago

Hi there, thank you for the great and detailed package!

I was wondering if you could help me resolve an error with the pipeline where the process is killed before STAR alignment is finished. I am running this on a cluster so maybe the issue is the reference to usr/bin/bash?

Building DAG of jobs... Using shell: /usr/bin/bash Provided cores: 4 Rules claiming more threads will be scaled down. Job counts: count jobs 15 Downsampling_HouseKeeping 1 STAR_matrix 15 Size_downsampling 15 align_bam_stat 15 bam_downsampling 1 batch_removal 15 cdr3_preprocess 1 deseq2_differential_genes 15 gene_body_cvg_qc 1 gsea_plot 15 index_bam 15 junction_saturation 1 merge_bcr_clonality 1 merge_bcr_infil 1 merge_bcr_process 1 merge_bcr_shm 1 merge_tcr_clonality 1 merge_tcr_infil 1 merge_tcr_process 1 pca_sample_clustering 1 plot_gene_body_cvg 15 read_distrib_qc 1 read_distrib_qc_matrix 1 salmon_matrix 15 salmon_quantification 15 ss_bcr_process 15 ss_tcr_process 1 ssgsea 15 star_align 1 target 15 tin_score 1 tin_summary 1 trust4_cohort_plot 15 trust4_repertoire 1 volcano_plot 245

[Tue Aug 9 14:44:38 2022] Job 23: Running STAR Alignment on s6234

STAR --runThreadN 64 --genomeDir /orange/brusko/leeanapeters/RIMA/ref_files/v22 --outReadsUnmapped None --chimSegmentMin 12 --chimJunctionOverhangMin 12 -$ Aug 09 14:44:38 ..... started STAR run Aug 09 14:44:38 ..... loading genome /usr/bin/bash: line 1: 9346 Killed STAR --runThreadN 64 --genomeDir /orange/brusko/leeanapeters/RIMA/ref_files/v22 --outReadsUnmapped No$ [Tue Aug 9 14:46:04 2022] Error in rule star_align: jobid: 23 output: analysis/star/s6234/s6234.unsorted.bam, analysis/star/s6234/s6234.sorted.bam, analysis/star/s6234/s6234.transcriptome.bam, analysis/star/s6234$ log: logs/star/s6234.star_align.log (check log file(s) for error message) shell: STAR --runThreadN 64 --genomeDir /orange/brusko/leeanapeters/RIMA/ref_files/v22 --outReadsUnmapped None --chimSegmentMin 12 --chimJunctionOverhang$ (exited with non-zero exit code)

Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: /orange/brusko/leeanapeters/RIMA/RIMA_pipeline/.snakemake/log/2022-08-09T144412.563019.snakemake.log 3.16user 87.63system 1:59.21elapsed 76%CPU (0avgtext+0avgdata 26050936maxresident)k 97291464inputs+592outputs (1major+6594049minor)pagefaults 0swaps

There is no output in the star alignment log.

Any help would be appreciated!

Leeana

Lindky commented 2 years ago

Hi, Thanks for your interest in the RIMA pipeline. When you get this "killed," that usually indicates you don't have enough memories or CPU if you use your local computer. But if you use a computing cluster or computing platform (GCP; AWS), please set up enough memory and CPU to run STAR alignment. In the tutorial, we recommend you at least use mom = 64 G and 32 cores. If you still get this error, you can improve the memory to 128 G.