De novo indexing using `--bam` capability

[timbitz]

This is a fairly major pull request with changes primarily affecting Whippet's indexing capabilities. This branch enables a user to supplement standard-annotation files with more sources of splice-sites for Whippet to utilize when indexing/building CSG nodes. These changes allow Whippet to produce more comprehensive CSGs in poorly annotated species, utilizing unannotated splice-sites (and therefore also unannotated exons) from read alignments in a pre-existing BAM file created by another program with novel spliced-read alignment abilities.

Specific changes are:

• Unannotated splice site support:
  • Sorted/indexed BAM files may now be supplied using the --bam parameter (whippet-index.jl)
  • Splice sites are extracted from BAM file in strand-specific manner. (refset.jl)
  • By default at least one splice-site in each spliced read must match a splice site from the annotation set: turn off this requirement with --bam-both-novel flag.
• CSG building improvements, which now utilizes both the annotated and unannotated splice sites from BAM. (graph.jl)
  • New nodes (can be either AA/AD/CE) can be built within sequence annotated as intronic for regions on the "match operation" side of each spliced-read. eg. upstream of novel 5' splice site, and regions downstream of novel 3' splice sites from spliced-alignments.
• Documentation README.md changes to guide building of indexes with --bam parameter.
• Alignment and CSG building bugfix:
  • No short overhangs past ->LS/SR<- edges.
  • SAM output always matches inferred strand from CSGA regardless of sequence orientation.
  • Splice-sites always override internal TxStart/TxEnd in CSG-building
• Deprecated: refflat format no longer supported--refflat flag (whippet-index.jl).
• New warnings for GTF files whose sequence names aren't found in the fasta file.
• New output: an .exons.gz file from whippet-index.jl that maps all putative exons in the Whippet index to the corresponding Whippet nodes and their annotation status.

Prior to merging, testing the PR should be possible with:

julia> Pkg.update()
julia> Pkg.checkout("Whippet", "de-novo-index")

Sorted and Indexed BAM files can be created with samtools as follows

$ samtools sort filename.bam filename.sort
$ samtools rmdup -S filename.sort.bam filename.sort.rmdup.bam
$ samtools index filename.sort.rmdup.bam
$ ls filename.sort.rmdup.bam*
filename.sort.rmdup.bam        filename.sort.rmdup.bam.bai

Then build an index but with the additional --bam filename.sort.rmdup.bam parameter, which also looks for filename.sort.rmdup.bam.bai

[codecov[bot]]

Codecov Report

Merging #69 into master will increase coverage by 4.52%. The diff coverage is 75.36%.

@@            Coverage Diff             @@
##           master      #69      +/-   ##
==========================================
+ Coverage   54.27%   58.79%   +4.52%     
==========================================
  Files          18       19       +1     
  Lines        2320     2378      +58     
==========================================
+ Hits         1259     1398     +139     
+ Misses       1061      980      -81

Impacted Files	Coverage Δ
src/edges.jl	72.58% <ø> (ø)	:arrow_up:
src/index.jl	10.29% <0%> (-0.64%)	:arrow_down:
src/quant.jl	59.32% <0%> (+2.62%)	:arrow_up:
src/graph.jl	93.69% <100%> (+0.18%)	:arrow_up:
src/types.jl	91.67% <100%> (+4.17%)	:arrow_up:
src/align.jl	71.63% <28.57%> (-0.5%)	:arrow_down:
src/io.jl	59.24% <72.5%> (+26.18%)	:arrow_up:
src/bam.jl	92.31% <92.31%> (ø)
src/refset.jl	92% <94.74%> (-1.37%)	:arrow_down:
... and 3 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update bed4f75...a7744b4. Read the comment docs.

[timbitz]