search

dasmoth / dalliance

Interactive web-based genome browser.
http://www.biodalliance.org/
BSD 2-Clause "Simplified" License
226 stars 68 forks source link

Randomly dissapearing bam tracks #187

Closed TyberiusPrime closed 8 years ago

TyberiusPrime commented 8 years ago

Hello,

I have an issue where tracks are fist rendered, then are replaced with empty screen estate in parts of their display. Let me illustrate.

This is the zoomed out view, everything is fine here (for now) zoomeed_out

I hit space, and for less than a second, this shows up (great) ok

to be rapidly replaced by first this half_broken

and at the end: this, where 3 out of 4 tracks are missing. broken

Sometimes zooming around a bit, or dragging dropping will restore the track, but it's at the point where it's seldom that you see all 4 of them at the same time...

This happens at any zoom stage, and having seen the track at one zoom stage is no guarantee that it will shop up after toggling with space twice: zoomed_out_broken

The console is empty, and this is with the github master from yesterday - though I upgraded because this was happening with the 0.12.0pre1 as well, which is why I upgraded.

Any ideas?

TyberiusPrime commented 8 years ago

If I read my debugger correctly, the records get loaded from the bam, then they get turned into features, those features are then sampled to 'density' features, which are then passed to tier.setFeature, das.prototype.draw sees them, passes them on to drawFeatureTier, which turns them into glyphs... which apperantly have an x value of -1000.something. Huh? But then the ones being drawn also start at -985. The glyphs do reach paintOnContext, they do have height, they're passod to glyph.draw...

perhaps they're beyign drawn of canvas?

TyberiusPrime commented 8 years ago

And why is paintOnContext being called twice? The rect is cleared both times, and the second round doesn't work.

TyberiusPrime commented 8 years ago

The issue seems to be independent of the drawing style used - changing it does not make the data show up.

TyberiusPrime commented 8 years ago

Ok, the first call to paintOnContext has subtiers (this.subtiers is an array of subtiers), the second one has only an empty array. When they're being drawn correctly (at another place in the coordinate system), subtiers is filled both times.

dasmoth commented 8 years ago

Sorry to hear you're having trouble with this. Is there a public/semi-public instance of this I could test?

(Feel free to respond off line -- thomas.a.down@gmail.com)

TyberiusPrime commented 8 years ago

Hello dasmoth,

I did so about a week ago - did you have a chance to check it out?

TyberiusPrime commented 8 years ago

I'm almost certain by now that the glyphs are being drawn outside of the canvas - just why, I have no clue yet. At least, when I manually fix a glyph with coordinate 1500+ to 940, I get a bar drawn.

TyberiusPrime commented 8 years ago

Attached a video of the effect, showing the track disappearing and re appearing multiple times. biodalliance_issue.zip

dasmoth commented 8 years ago

I'm still having extreme difficulty reproducing this in my development environment (but do recognize there seems to be a problem here somewhere).

One thought: if you haven't already done so, could you try:

       maxWorkers: 0

in your configuration?

TyberiusPrime commented 8 years ago

That does not change the behavior :(.

I've attached a stripped down bam that has reads from a 10 megabase window and still shows the issue on my setup. Look at region 7:95,220,652..95,324,193 for the effect: debug.zip

I can also see the issue when using the browser currently at Biodalliance.org (Saved the file, added a source, loaded the result from another webserver).

dasmoth commented 8 years ago

Many thanks for sharing this file -- problem now reproduced in a simple test setup at my end. Excludes a bunch of possibilities.

The odd thing is that I tried reproducing the problem with and identical setup and several other BAMs this morning, and got nowhere. That's at least suggestive that there's something odd going on with the file itself... What tools (and versions) did you use to create the file?

TyberiusPrime commented 8 years ago

The original alignment was done with Subread (1.4.3-p1)- the resulting SAM file was converted with pysam. Pysam (git commit fa8e29e0cb0d) also did the sorting and indexing - but that get's passed on to samtools I believe.

The fragment I sent you was filtered with pysam as well.

dasmoth commented 8 years ago

Okay, I have a clue: if I reindex your debug.bam file using samtools 0.1.19 (nothing particularly special about that choice of version, it's just what happens to be on the machine on my desk right now), I was able to scroll through the entire 10Mb region without seeing any missing chunks. My index is attached if you want to experiment yourself debug-index-dasmoth.zip. I didn't make any changes to the BAM file itself, so have only attached the new indices.

I haven't had a chance to poke into the differences between the two indices, but there clearly are some -- the new index is 67784 bytes long, vs. 51864 bytes for the original.

If you get a chance, could you re-index one of your files using a different tool (maybe samtools, if available) and see if that helps?

TyberiusPrime commented 8 years ago

I can reproduce that the problem disappears with samtools 0.1.19 reindexing of this particular file - it also seems to be working with the full BAM.

Reindexing with a current pysam from a fresh pip install leads to the issue .bai again.

dasmoth commented 8 years ago

It seems there was indeed a bug here, but only exposed with index files generated by tools which try quite aggressively to merge chunks in the index.

Fixed now in git master.