search

nservant / HiC-Pro

HiC-Pro: An optimized and flexible pipeline for Hi-C data processing
Other
386 stars 182 forks source link

Read block operation failed in only one read pair #376

Closed Rayko87 closed 3 years ago

Rayko87 commented 4 years ago

Hi Servant,

I managed to run your pipeline succesfully in the past, but now it stops in the mergeSAM step. After 16 days of computation with no errors in the log, I saw this in the mappingstats.log file:

mapping_stats_log /usr/bin/samtools view -c bowtie_results/bwt2/CUTL3/CUTL3-HiChIp_R1_001_GRChg37-hg19.bwt2merged.bam /usr/bin/samtools view -c bowtie_results/bwt2/CUTL3/CUTL3-HiChIp_R2_001_GRChg37-hg19.bwt2merged.bam [E::bgzf_read] Read block operation failed with error -1 after 0 of 4 bytes [main_samview] truncated file. /usr/bin/samtools view -c -F 4 bowtie_results/bwt2/CUTL3/CUTL3-HiChIp_R2_001_GRChg37-hg19.bwt2merged.bam /usr/bin/samtools view -c -F 4 bowtie_results/bwt2_global/CUTL3/CUTL3-HiChIp_R2_001_GRChg37-hg19.bwt2glob.bam

However, as you see, the R2 is nicely processed, but not the R1 (This R1 is 33Gb big and the R2 is 30Gb). Consequently, in the next log file it is not finding both R1 and R2:

MergeSAM.log

/usr/bin/python /usr/local/bin/HiC-Pro_2.11.1/scripts/mergeSAM.py -q 10 -t -v -f bowtie_results/bwt2/CUTL3/CUTL3-HiChIp_R1_001_GRChg37-hg19.bwt2merged.bam -r bowtie_results/bwt2/CUTL3/CUTL3-HiChIp_R2_001_GRChg37-hg19.bwt2merged.bam -o bowtie_results/bwt2/CUTL3/CUTL3-HiChIp_GRChg37-hg19.bwt2pairs.bam

mergeBAM.py

forward= bowtie_results/bwt2/CUTL3/CUTL3-HiChIp_R1_001_GRChg37-hg19.bwt2merged.bam

reverse= bowtie_results/bwt2/CUTL3/CUTL3-HiChIp_R2_001_GRChg37-hg19.bwt2merged.bam

output= bowtie_results/bwt2/CUTL3/CUTL3-HiChIp_GRChg37-hg19.bwt2pairs.bam

min mapq= 10

report_single= False

report_multi= False

verbose= True

Merging forward and reverse tags ...

Forward and reverse reads not paired. Check that BAM files have the same read names and are sorted.

Any clue why this is happening? Doing this command separately /usr/bin/samtools view -c bowtie_results/bwt2/CUTL3/CUTL3-HiChIp_R1_001_GRChg37-hg19.bwt2merged.bam giv es the same error than in the log (but only R1, not R2).

Thanks! Robert

nservant commented 4 years ago

Hi Robert, Which version are you using please ? Are you sure that the original fastq files are ok ? Thanks

Rayko87 commented 4 years ago

Hi Servant,

Thanks for your answer. The samtools version is:

Program: samtools (Tools for alignments in the SAM format). Version: 1.7 (using htslib 1.7-2)

About checking the original fast....how would you recommend to check them?

Thanks, Robert

nservant commented 4 years ago

Could you try to see where exactly the R1 was being corrupted ? You can try to look at the different logs, and intermediates mapping files. My feeling is that the error is due to something wrong during the computation (RAM / disk issue ?)

However, it is still a good idea to double check that you your fastq files are ok, well paired, with the same number of reads