Open wm75 opened 5 years ago
@wm75 I am having difficulties reproducing this behaviour. Could you provide a relevant anonymized subsample of your data?
Sure, no problem. The data in mapped_data.zip is the relevant (for producing the reported output) snippet (chr1:248567000-248568000) from publicly available test data from the Texas Cancer Research Biobank (sample TCRBOA6-N). There is a rather extreme extent of read overlaps in this dataset so possibly it's only then that the issue becomes noticeable. Thanks for looking into this and let me know if there's anything else I can do to help you.
@wm75 thank you for the data! We would have struggled long and hard to reproduce the issue without it.
First, when you specify a mpileup window (with -r), reads that do not overlap it are discarded from the mpileup computation, which should not affect the output, as those reads had no influence over the selected region. Second, by default, mpileup discards the overlapping segment of pair-end reads. You can suppress this behaviour using -x flag.
Now, focusing on 3 particular reads from this BAM file:
HWI-ST896:350:D23CTACXX:7:2113:14780:50737 2115 chr1 248567484 7 13M6D34M = 248567503 19
HWI-ST896:350:D23CTACXX:7:2113:14780:50737 147 chr1 248567484 22 25S13M6D44M16S = 248567503 -44
HWI-ST896:350:D23CTACXX:7:2113:14780:50737 99 chr1 248567503 60 3S44M51S = 248567484 44
we can see the first is a supplementary read (flag 2048 set), which also happens to end at coordinate 248567536. The other two are pair-end reads.
When you specified the mpileup region as chr1:248567536-248567542, the first read was included in the computation and also in the overlap removal logic, because it was seen as part of the same template as the other two reads (they all share the same template name). This was affecting how reads were paired and which overlap segment was being removed. With the -x flag on, the overlap removal was not being performed, so the supplementary read was not affecting the results.
On the other hand, when the selected region was chr1:248567537-248567541, the first read was discarded from the beginning, and the overlap removal algorithm only used the pair-end reads (2 and 3), as it was designed. A fix will follow soon.
Thanks for the investigation and the detailed explanation @valeriuo ! Glad that I could help.
I've observed in several cases that changing the region (
-r
option of smatools mpileup) in which a pileup is generated affects the depth reported at a fixed position. One example from real data:$samtools mpileup -B -r chr1:248567537-248567541 -f hg19.fa mapped_data.bam
now increase the width of the region by one base on each side:
$samtools mpileup -B -r chr1:248567536-248567542 -f hg19.fa mapped_data.bam
and observe how an additional read gets included in the pileup at positions 37-41.
Playing around with the different
samtools mpileup
options to see what's causing these discrepancies I found that they disappear when you disable read-pair overlap detection with the-x
option, e.g. same calls as above, but with overlap detection disabled (this particular dataset has lots of overlaps):$samtools mpileup -B -x -r chr1:248567537-248567541 -f hg19.fa mapped_data.bam
and
$samtools mpileup -B -x -r chr1:248567536-248567542 -f hg19.fa mapped_data.bam
My interpretation of this behavior (not having studied the actual code) is that the overlap detection does not rely on the entire read-pairs, but somehow only on the part of them, which falls inside the requested region?
So first of all, I would like to understand what exactly the interaction between the
-r
and-x
options is. Secondly, at least intuitively, this looks like a bug in samtools mpileup to me (after all, you wouldn't expect the depth reported for any particular base to be dependent on the window through which you look at it).