Provide extensible parsers for user-provided data

[abought]

Ticket

237

Purpose

The existing lz-tabix-source addon provides a way to load generic tabix files, but it does not handle the act of parsing. We will add basic parsers for several line based formats. If time permits, we should also explore UI (an add-track widget) that comes pre-configured with common options.

The "add track" widget will focus on tabix-based files, rather than generic text files. The rationale for this is that some users might have very large datafiles that would overwhelm the browser. Using a tabix query will prevent a bad user experience (browser crash), though at the cost of some minor added effort during data preparation.

TODO

• Parser support for several formats. How much shared configuration is necessary?
• To add in this release
  • [x] BED
  • [x] PLINK-LD
  • [x] GWAS parser - migrate from LocalZoom
• Ideas triaged for future use as needed
  • [~] VCF
  • [~] BEDPE
  • [~] GFF(version??)
  • [~] Compare to IGV.js implementation for common edge cases: https://github.com/igvteam/igv.js/blob/master/js/aed/AEDParser.js
• Validation / sniffer logic
  • [x] Sample files seem to date often deviate from the BED spec in odd ways: test the BED parser thoroughly on a variety of real-world formats (with especial focus on the type-coercion behavior: will it break on common deviations?)
• Usability
  • Explore an "add track" helper function similar to IGV.js, exposing the most common options that people might wish to use (MOVE TO FUTURE)
  • Known issues
  • [x] LDrefvar should be removed from URL if variant is outside the requested range. Filtering it at the adapter level breaks the UserTabixLD adapter, due to recursive method call on different class.

[abought]

Contributions to release notes tracked below

• New tools for adding user provided data to the plot:
  • lz-parsers extension provides helpers to read BED, GWAS, and PLINK user-provided LD files
  • New tabix_tracks demo showcases how to use these parsers with data on a remote S3 bucket, without transforming files into an alternative JSON representation first. Instructions are provided for calculating custom LD.
  • New bed_intervals panel and data layer layouts make it easier to use standard-format BED file data with the existing intervals track
• Build process was changed to facilitate easier use of extensions
  • gwas-credible-sets and tabix-reader libraries no longer need to be included separately when using related LZ extensions. The extension also automatically incorporates its dependencies.