Release/0.14.0

[abought]

This PR represents all changes tagged for release 14, which will emphasize support for user-provided data, including improved tabix support and built-in parsers for common data formats.

Since there are several large chunks to this release, this PR gathers release notes and additional required tasks in one place.

Release tasks:

• [x] Public testing period via LocalZoom / my.locuszoom.org
• [x] Update documentation and tutorials to reflect the new data / request syntax.
• [x] Consider moving undercomplicate back into LZ core (it started as a separate library to facilitate prototyping)
• [x] Gather release notes from various subtasks
• [x] Showcase the tabix tracks demo in several places for new users
• [x] Finalize release notes

[abought]

Release notes will be tracked in this comment.

This release makes many internal changes to how LocusZoom retrieves data. It provides new functionality that should make it easier to create custom plots from various types of file. Some minor display changes have also been added to improve the appearance and readability of figures.

New features

• Built in parsers and a new "Tabix Tracks Demo" make it easier to generate plots from tabix files for GWAS, BED, and user-provided LD, without manually creating a JSON file or loading the data into an API first. This should make it easier to get started with small in house datasets!
• Developer-focused internal changes make some aspects of generating plots less confusing.
• Both GWAS and PheWAS layouts will now try to render points with an up/down triangle to indicate direction of effect, when possible. Internally this uses the new effect_direction ScaleFunction, inspired by PheWeb.
• Default plot font sizes have been increased by 20-40%, to help readability in publications/presentations
• New more compact LD "ribbon" legend
• (from LZ 0.13.x) New "turbo" color scheme provides better support for colorblind users (or black and white printing)

Internal improvements

• Plots can now switch between different view regions more quickly, thanks to a new multi-value LRU cache that can eliminate unnecessary network requests
• TabixUrlSource now accepts a pre-made reader instance . This is useful for sites that want to validate file contents before adding the file to the plot.
• Data layers now accept a template expression for id_field. This means that data elements can be uniquely identified as a combination of several fields. This eliminates the need to add "synthetic" unique ID fields in plots like phewas or coaccessibility, where no single datum field would have been globally unique. This also means that certain tooltip and mouse behaviors will be more stable as the user pans and zooms.

Deprecations and major breaking changes

We generally try to avoid issuing breaking changes, but after 5+ years of LocusZoom.js, we are doing some tidying to make custom and interactive usages easier. Please consult the updated documentation and guides, and let us know if any important information is missing after the rewrite!

Layout syntax

• Namespaces have been redesigned as the central mechanism for plotting two datasets with one reusable layout.
  • The difference between "abstract" and "concrete" layouts has been eliminated. Syntax like Layouts.get(..., {unnamespaced: true}) or some_value: '{{{{namespace[assoc}}afield}}' is no longer necessary. All layers will now refer to fields by an internal name that is easier to read, write, and remember: assoc:afield.
  • Useful migration regex: replace \{\{namespace\[(\w+)\]\}\} with $1:
• New data_operations make it possible to mix-and-match different datasets, by moving the join logic from adapter to the point where the data is used (the data layer).
  • This replaces the old combineChainBody method of adapters. Closely coupling the act of retrieval (API) to the join logic (display) made it hard to use data fetched from a different location.
  • This also makes it much easier for a single API call to be displayed in several different ways throughout the plot.
• The data_layer.fields array property has been removed completely. Many people wanted to customize their data with just one extra field, and manually updating the contract of expected data proved cumbersome.
• Panel layouts must now provide an explicit id (no longer optional). Most layouts already did this.
• subscribeToData call signature has changed. It now accepts either an explicit specification of what data is requested, or else the name of a data layer ("give me exactly what is being used to draw something that is already specified elsewhere"). See docs for details.
• Removed old aliases for things that were renamed in prior versions: all references to dashboard.components now use toolbar.widgets

Adapters

• Adapters have been completely rewritten in a more functional style, making them easier to understand and debug. Many methods have been renamed slightly to reflect that they have a different call signature. Almost all custom adapter subclasses will need to update their code.
  • Example: getURL(state, chain, fields) is now _getURL(request_options); annotateData(records, chain) is now _annotateRecords(records, request_options), etc.
• The concept of the data chain has been eliminated; it was confusing and made it hard to use the same data in two different ways.
  • New "data operations", like "joins", specify how to connect two datasets at the point where the data is used, rather than when the data is first retrieved. The adapter method combineChainBody has been totally removed in favor of data_layer.data_operations. Several join functions are provided by default.
  • If your request depends on a prior request (like "LD for a particular GWAS variant"), a new dependency resolver has been introduced. See the guides/tutorials describing data_operations: [{type: 'fetch', from: ['assoc', 'ld(assoc)']}] for details of how to specify dependent requests.
• Adapter options are now specified as top-level keys, instead of inside a nested key calledparams: {}. Existing sample code to create a plot will need to revise usages of DataSources.add().
• Adapters will usually return all the fields from the API.
  • All pre-defined data layer layouts now contain an _auto_fields variable that defines what information a layer expects. At the time of render, a user-friendly error message will warn if the server fails to provide expected fields.
  • If you use LD information, rename ld:isrefvar to lz_is_ld_refvar and ld:state to ld:correlation. The new names for "auto-generated" properties better reflect what is actually coming back from the API, and make the code far easier to understand.
• Any data source specified in data_layer.namespaces will now trigger a network request for data. This is different than the previous behavior, where a different property ("fields") had to also be mutated
  • People really liked adding custom fields/ data to the plot, but no one ever remembered to update data_layer.fields. This lead to a lot of confusion when their data failed to appear.

Other

• Many private variables were renamed in LocusZoom plot and panel internals. This should make the console debugging experience nicer, by hiding things that were not useful to most developers.

Bugfixes:

• If the user selects an LD refvar, ignore the choice when switching to another region far from that variant.
• Cached adapter data now does a better job of isolating each individual layer that uses the data. This is especially noteworthy in, eg, StaticSource
• Fix an issue where plot would be sized incorrectly when initially drawn in a hidden area (like collapsible or tab widgets). Plot will now resize to fit available space when the area is un-collapsed.
• Fix broken rendering of intervals (eg BED tracks) if they had very narrow markings
• Increase spacing around axis labels (so that plots look good with logp > 1000; previously, 4-digit numbers overlapped with the label)