LincolnSteinLab / gdc-viewer

A JBrowse plugin for viewing GDC Data

https://lincolnsteinlab.github.io/gdc-viewer/

MIT License

3 stars 2 forks source link

Use GraphQL to determine some useful metrics regarding index files #91

Closed agduncan94 closed 3 years ago

agduncan94 commented 3 years ago

It would be very useful to know the following: BAM index

how many BAM files have index files
how many BAM files DO NOT have index files
which BAM files have index files
which BAM files DO NOT have index files

VCF index

how many VCF files have index files
how many VCF files DO NOT have index files
which VCF files have index files
which VCF files DO NOT have index files

VCF gzip

how many VCF files are gzipped
how many VCF files are NOT gzipped
which VCF files are gzipped
which VCF files are NOT gzipped

Each header should be achievable in a single request. Use the Chrome GraphIQL extension for playing around with these.

agduncan94 commented 3 years ago

Based on a discussion with GDC, all BAM files available through the portal should have index files.

GFJHogue commented 3 years ago

Figured out how to use filtering & aggregations in the GDC GraphQL endpoint. Documentation is sparse, but it's powerful once learned!

BAM index

how many BAM files have index files: 74018
how many BAM files DO NOT have index files: 5119 (6.5%)
which BAM files DO NOT have index files:
- all 5107 BAM files with the STAR 2-Pass Transcriptome Workflow Type
- 12 BAM files, WXS, across 6 particular cases within the CPTAC-2 project.

GFJHogue commented 3 years ago

VCF index

how many VCF files have index files: 63336
how many VCF files DO NOT have index files: 102700 (62%)
which VCF files DO NOT have index files:
- all 66690 TCGA program VCFs with Workflow Type being one of SomaticSniper (Annotation), MuTect2 (Annotation), or MuSE (Annotation)
- all 36008 FM program VCFs
- 2 VCFs from 1 case in the TARGET-WT project

GFJHogue commented 3 years ago

VCF gzip

This was more difficult to query. There is no functionality in the GDC GraphQL from what I could tell that would allow me to examine file-endings for .vcf vs .vcf.gz. The following findings are based on manual exploration of the GDC repository:

how many VCF files are NOT gzipped: at least 11109 (6.7%)
which VCF files are NOT gzipped:
- VCFs in the TCGA program with Workflow Type MuSE (not the annotations)