Internal cleanup: ES6 + Webpack + D3v5

[abought]

💣 This PR includes some breaking changes to conventional LZ usage, especially for users who haven't upgraded in a while.

Internal changes

1. Convert internal code organization to ES6 modules and classes
2. Replace legacy gulp build system with webpack + babel
3. Move many "internal" features out of the global namespace
4. All "plugin" features are now based on ES6 classes, making it easier to customize small pieces of functionality
5. LocusZoom is now based on D3v5. If you are using the library via a script tag, you will need to update the D3 version used. We no longer provide a vendor.js bundle.

New features

• More systematic way of registering and bundling LZ plugins (LocusZoom.use)
• New "save as PNG" feature (and more reliable "save as svg" button- no longer relies on a hardcoded CSS file name)
• New filtering mechanism to restrict the points shown on scatter, arc, forest, and gene tracks, based on customizable criteria. (including UI toolbar widget to change filters interactively)
• New plot.destroy() method makes it easier to clean up global event listeners. (useful for component-based frameworks where plots may be created and destroyed)
• New region_changed event reports exact plot coordinates after pan, zoom, or region change
• New log10 transformation function (useful for creating eg loglog plots)

New/ improved defaults

• The "select LD population" button is added to all association plot toolbars by default
• Genes track changes allow more condensed rendering
• Removed responsive_resize: both mode as a plot layout configuration option. The default option (true) will now trigger resize only for width. Preserving aspect ratio sometimes caused odd behavior on rerender, and many of LZ usages will have already migrated to the new default.
• Fix "mouse scroll to zoom" in firefox (shift-scroll was used by the browser; provide alt-scroll as a second alternative)
• Build 38 users will receive LD information from an improved 1000G LD panel based on deep WGS

Potentially breaking changes

• The old LDLZ data source (adapter) has been completely removed, because it referenced a legacy server. Prior LocusZoom versions had already introduced introduced its replacement, LDLZ2 (also known as LDserver).
  • The "LD population selector" button has been added to all region plot toolbars by default. This button must be used in tandem with the LDserver adapter.
• Any methods previously marked as deprecated in prior releases have been removed. In previous versions, these were usually indicated by warning messages on the JS console.
• To resolve confusing nomenclature, several things have been renamed:
  • Internal registries renamed:KnownDataSources -> Adapters; Data.Source has been renamed to BaseAdapter / RemoteAdapter, and is no longer available as a global.
  • The helper used to assign data to a specific plot (LocusZoom.DataSources) was left unchanged.
  • Dashboards have been renamed into "toolbar" and Dashboard.Components -> Widgets. This rename may affect custom layout code, eg dashboard.components should be changed to toolbar.widgets and Layouts.get('dashboard'...) -> Layouts.get('toolbar'...).
• All registries are now more consistent in their behavior, exposing add, remove, has, get, and list methods. Efforts have been made to preserve method syntax for any existing usages.
• All user-extensible types are exposed as true Es6 classes, allowing greater use of custom code (eg for new data layers).
  • Subclasses are now created via ES6 class syntax. To create a new user-defined data or rendering class, the various (inconsistent) extend methods have been replaced by someregistry.add(es6Subclass).
• The LocusZoom.createCORSPromise method has been removed. Internal usages have been replaced with the standard JS method fetch; we recommend doing the same with your own code.
• The line layer interpolate option now uses the names for d3.line.curve
• Removed setElementStatusByFilters and related methods
• Remove datalayer.addField method, due to limited utility. It is better to know what fields are required before requesting the data.

Known issues (in progress)

• [x] Dashboards may behave oddly in dev vs prod builds
• [x] Scatter plot lines/labeling is broken for phewas plot demo on second render. (should be first render- adjust merge selection)
• [x] Save as png button- firefox rendering issue
• [x] LD source should handle TOPMED-style variant identifiers when normalizing variant IDs

TODO

• [x] Upgrade to D3v5
• [x] Trim margins visually and panel heights throughout to increase information density
• [x] Track event listeners and implement a proper plot destructor
• [x] Remove responsive_resize: both mode
• [x] Investigate LD panel enhancements
• [x] Add KnownDataSources.extend for convenience with creating new datasources without reaching into registry (targeting Es5 environments)
• [x] Clean up some legacy function signatures for deprecated syntax (old layout conventions, etc)
• [ ] Improve JSDoc builds and migrate guides to jekyll. (how will things like filters be documented?)
• [x] Incorporate late breaking LZ 0.11 improvements; rebase to develop
• [x] Tweak plugin behaviors to prevent double-install
• [ ] #199 , if API is ready in time

[abought]

Draft release notes below

This version contains numerous breaking changes and new features. Although we have tried to keep most familiar aspects of using LocusZoom intact, please read carefully before upgrading.

If you are using LocusZoom via <script> tag, LocusZoom now requires D3v5. Check any other visualizations on your site for compatibility.

New features for users

Several new features are available and enabled by default.

• New "save as PNG" feature
• Fix "scroll to zoom" in Firefox (new "alt-scroll" keyboard shortcut)
• Better LD by default
  • Build 38 datasets will now use an improved LD panel based on deep WGS sequencing of 1000G.
  • The "select LD population" button has been added to the default layout for all association plot toolbars (this button must be used with the LDServer adapter)
• Improved genes track
  • Now hides pseudogenes by default.
  • A dropdown menu can be used to customize what is shown.
  • Visual tweaks: Genes track now shows more genes in the same amount of space (better for publications etc)
• Gene and GWAS catalog views refreshed with the newest data available (GENCODE v35 and GWAS catalog v1.0 e100)

New features for developers

LocusZoom is highly customizable. We have added a variety of internal improvements that make it easier to build truly dynamic visualizations.

• New "filters" mechanism lets users interactively choose which points are shown on scatter, arc, forest, and gene tracks, based on customizable criteria.
  • A new toolbar widget can be used to let users change filters interactively
• The "save as svg" button no longer relies on a hardcoded CSS file name. This should resolve issues where saved images did not look good on some sites.
• New region_changed plot-level event reports exact plot coordinates after pan, zoom, or region change
• New log10 transformation function (a complement to neglog10). Useful for creating, eg, loglog plots.
• New plot.destroy() cleanup method: helps to avoid memory leaks when using LocusZoom with single-page-application frameworks like Vue or React.
• More systematic way of registering and bundling LZ plugins (LocusZoom.use(function(LocusZoom, ...args) { /* plugin code*/ }))
• All registries (layouts, adapters, datasources, etc) are now more consistent in their behavior, exposing add, remove, has, get, and list methods. Efforts have been made to preserve method syntax for any existing usages.

Important breaking changes

The code was heavily reorganized. Unused features were removed, and some improvements were made to make the library more approachable for new developers.

Overview

• LocusZoom is now based on D3v5. If you are using the library via a script tag, you will need to update the D3 version used. We no longer provide a vendor.js bundle.
• All extensible "plugin" features are now based on ES6 classes, making it easier to customize small pieces of functionality (like new data layers)
• The codebase is now based on ES6 modules. Many "private" features have been moved out of the global namespace, to avoid overwhelming developers with functions they don't need.
  • LocusZoom will only be exposed as a global variable if you load it via <script> tag.
  • If your code uses ES6 modules, import LocusZoom from 'locuszoom'; will continue to work. But you should update how you import extensions for best results: use locuszoom/esm/ext/... instead of locuszoom/dist/ext/...

Renamed features

To support doc/instruction improvements, several things have been renamed.

• Three different things had "data source" in their names and all were available globally. This was unnecessarily confusing.
  • LocusZoom.KnownDataSources -> LocusZoom.Adapters
  • Data.Source has been renamed to BaseApiAdapter and is no longer available as a global.
    • Users of modern ES6 may import it from locuszoom/data to create true ES6 subclasses.
    • If using via a <script> tag, the shortcut for creating and registering a new class will be almost the same as old versions: LocusZoom.Adapters.extend('BaseApiAdapter', 'MySourceName', {}) .
  • The helper used to assign data for a specific plot (LocusZoom.DataSources) was left unchanged.
    • dashboards are now called toolbars , and Dashboard.Components -> Widgets.
  • This rename will affect layouts: eg dashboard.components should be changed to toolbar.widgets and Layouts.get('dashboard'...) -> Layouts.get('toolbar'...).

New/improved defaults

• Plots now handle resizing differently. The old responsive_resize: both mode has been removed. The default option (true) will now trigger resize only for width.
  • Preserving aspect ratio sometimes caused odd behavior on re-render. Many sites that use LZ have already migrated to the new default.
• Layouts (such as annotation tracks) that used filters now have a new unified syntax, where each filter is represented as an object with {field, operator, value} keys. This extends and standardizes the quasi-public release of track filters from previous releases.
  • Some old layouts represented each filter in an array form and will need to be updated.

Removals

• Any methods previously marked as deprecated in prior releases have been removed. In previous versions, these were usually indicated by warning messages on the JS console.
• The LocusZoom.createCORSPromise method has been removed. Internal usages have been replaced with the standard JS fetch function; we recommend doing the same with your own code.
• The old LDLZ data source (adapter) has been completely removed, because it referenced an old server. Prior LocusZoom versions had already introduced its replacement, LDLZ2 (also known as LDServer).
• Data.Source.extend has been removed, in favor of true ES6 classes.
• Removed setElementStatusByFilters and largely unused methods.
• Remove datalayer.addField method, due to limited utility. It is better to know what fields are required before requesting the data.
• Removed "transition" option from layouts, which was poorly supported and largely unused.

Other

If you are testing experimental LocusZoom features, we have changed the build scripts. Gulp has been removed and replaced with new commands: npm run dev and npm run build. The unit tests now support test coverage.

The build process and modules work on all currently supported node LTS versions: 10.x or above. Old versions of Ubuntu tend to ship with very old versions of nodeJS, so this is something to check if your modified locuszoom code refuses to build.

If you modify the code, we welcome open source contributions! Please share your improvements or suggestions.