Unrecognized index type error after update

[fwet]

Hi- I just updated to latest igv.js and I'm getting this error (see screenshot below).

This tabix index is not a directly served file by the web server, it's processing the request range headers server side and serving the bytes for that range. This worked before in a previous version of IGV.js so not sure what exactly changed (apparently quite a few things)

Previously I ended up setting the parameters url, indexURL, and filename in the VCF track config because igv.js had issues deciphering a url with ? and params, rather than just a plain filename.

It seemed like this change I suggested was incorporated into a release, then possibly there was a regression or a refactor (i posted that bug to igv.js on an old github account, long story short that account does not exist any more)

Hoping this may ring a bell and you have some suggestions to debug this.

[jrobinso]

I don't understand your setup, you say the "tabix index is not a directly served file...". If you set indexURL its going to expect a tabix (or tribble) index. Of course igv.js is just making requests for bytes to the server, whether that's backed by a file or not is not relevant, but the bytes returned would have to conform exactly to the tabix spec. If you're not using an actual tabix index maybe a custom reader or webservice would work better.

If you give me a test case I an reproduce I will look into this, but first could you describe what you are going a little more completely. And if there is previous git issue relevant to this include a link to it.

[fwet]

I believe it does conform to the spec, and it worked fine until the upgrade.
Last time there was an issue with the file names being as follows:

/files/bs?f=2e9be1d69785f8209a15f40675cbd93a.tbi and /files/bs?f=1b3c66905e8d63d7e59311b756e1d155.gz

vs

myfile.vcf.gz and myfile.vcf.gz.tbi

Hoping it's something that simple again but I'll see if I can narrow it down, if nothing jumps out to you immediately from reading this..

The error comes from switch (magic) in indexFactory.js loadIndex() so I'll check on that.

Thanks

[jrobinso]

Are you sure the magic number returned is correct? The type of index (bam, tabix, or tribble) is determined from the magic number, I think in the past the extension was used. The filenames should no longer matter, as long as you are setting the format property in the config to "vcf".

On Sun, Jan 3, 2021 at 9:10 PM fwet notifications@github.com wrote:

I believe it does conform to the spec, and it worked fine until the upgrade. Last time there was an issue with the file names being as follows:

/files/bs?f=2e9be1d69785f8209a15f40675cbd93a.tbi and /files/bs?f=1b3c66905e8d63d7e59311b756e1d155.gz

vs

myfile.vcf.gz and myfile.vcf.gz.tbi

Hoping it's something that simple again but I'll see if I can narrow it down, if nothing jumps out to you immediately from reading this..

The error comes from switch (magic) in idexFactory.js loadIndex() so I'll check on that.

Thanks

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/igvteam/igv.js/issues/1251#issuecomment-753759361, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHD2HCKSL7CT2UVXQOXHTTSYFEWBANCNFSM4VSOKBEQ .

[fwet]

The magic numbers seem correct- Will let you know what I find.

This BAM and BAI that I loaded directly from the file system (bypassed the proxy w/ range headers) into igv.js also produced the error "Unknown file type: test-mapped-sorted-mkdup.bam"

It's from pretty standard mapping piepline, bwa, samtools- I think gatk or picard mark duplicates is in there at the end. Same pipeline I've been using.

Unix file command checks magic number:

file test-mapped-sorted-mkdup.bai test-mapped-sorted-mkdup.bai: SAMtools BAI (BAM indexing format), with 193 reference sequences

file test-mapped-sorted-mkdup.bam test-mapped-sorted-mkdup.bam: Blocked GNU Zip Format (BGZF; gzip compatible), block length 18923

This file does have the BAM magic but it's inside the bgzip, so maybe it's reading bgzip magic first? I can see that the loadIndex() function does detect bgzip and deflates it first. So it should handle this.

I'll add some debugging output to the igv.js code and see what else I can find. Didn't get a chance yet, but I'll test it against some of your test files.

gunzip -c test-mapped-sorted-mkdup.bam | less BAM^A<9F>^\^@^@@HD VN:1.6 SO:coordinate

[jrobinso]

See the examples at https://igv.org/web/release/2.7.4/examples and https://igv.org/web/release/2.7.4/examples/test. Also, igv.org/app uses igv.js 2.7.4 and there are bam and vcf files you can load from the "Tracks" menu for hg19.

Can you create a reproducible test case I can run?

[fwet]

One issue solved- The prior verison would accept track config

'type':'bam'

whereas now it needs:

'type':"alignment" 'format':'bam'

Now it loads my bams again properly.

I'll see what's up with the remaining stuff and report back, if you want to leave this issue open for a while longer. Thanks.

[jrobinso]

The track type is "alignment", format is "bam". See the documentation.

On Thu, Jan 7, 2021 at 5:58 PM fwet notifications@github.com wrote:

One issue solved- The prior verison would accept track config

'type':'bam'

whereas now it needs:

'format':'bam'```

Now it loads my bams again properly.

I'll see what's up with the remaining stuff and report back, if you want to leave this issue open for a while longer. Thanks.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/igvteam/igv.js/issues/1251#issuecomment-756497840, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHD2HHTNCF4QSFMC5KI723SYZRDRANCNFSM4VSOKBEQ .

[fwet]

Seems like all my downstream bugs were just related to the above. The old version was able to handle my misconfiguration of the track whereas the new version wasn't, probably due to no longer relying on file extensions? At any rate it's working now.

Thanks for your help. Closing this issue.

[jrobinso]

I'm not really sure, but glad it is working.

On Thu, Jan 7, 2021 at 10:23 PM fwet notifications@github.com wrote:

Seems like all my downstream bugs were just related to the above. The old version was able to handle my misconfiguration of the track whereas the new version wasn't, probably due to no longer relying on file extensions? At any rate it's working now.

Thanks for your help. Closing this issue.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/igvteam/igv.js/issues/1251#issuecomment-756573964, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHD2HGCXWJU6LOXRFQDZ7LSY2QE5ANCNFSM4VSOKBEQ .