EOF error during sort_index_stats

NCI-RBL / iCLIP

RNA Biology Pipeline to Characterize protein-RNA Interactions

https://rbl-nci.github.io/iCLIP/

MIT License

4 stars 2 forks source link

EOF error during sort_index_stats #112

Closed slsevilla closed 2 years ago

slsevilla commented 2 years ago

Running Soyeong's samples through the new pipeline has lead to a problem with one sample. YKO_Clip

[W::bam_hdr_read] EOF marker is absent. The input is probably truncated

samtools sort: truncated file. Aborting

error file here: /data/RBL_NCI/Wolin/mESC_clip_3_clip3_v2.0/log/20220530_1802/06_sort_index_stats.40723673.sp=YKO_Clip.err

slsevilla commented 2 years ago

I read about this error and a lot of posts pointed to the alignment. I went back through and deleted all of the intermediary files and repeated alignment, but I'm still getting the exact same error. Other samples have completed without an issue so it doesn't seem to be pipeline specific but I can't figure out what is going on here.

Going to attempt to implement RSYNC to see if it's an issue with file transfer from /lscratch/ to /output/dir.

slsevilla commented 2 years ago

In talking with HPC staff, I was suggested to increase LSCRATCH during the STAR alignment step, and then process samples again. I have implemented this.

star: time: 04-00:00:00 gres: lscratch:800

slsevilla commented 2 years ago

point tmp files from STAR to lscratch with commit 8519b0dad1d682a9825d28803b69b71484414918
STAR files are already sorted - remove sorting from code 7f7c6237365724e15762e2cabb742791c0359bc3
increased LSCRATCH space with commit bf1bae4883184cb9169c13f9f39c4a8bf09577bf

slsevilla commented 2 years ago

SOLUTION: increasing LSCRATCH allowed for STAR to complete alignment, and downstream steps to complete without EOF error. This was already committed to pipeline so no subsequent work needed (bf1bae4883184cb9169c13f9f39c4a8bf09577bf).