Allow hiding empty grid columns to improve legibility of large sparse grids

[wbazant]

Related to #1148. @cmdcolin

I have this grid: http://test.parasite.wormbase.org/jbrowse/index.html?data=%2Fjbrowse-data%2Fcaenorhabditis_elegans_prjna13758%2Fdata&loc=X%3A937771..957837&tracks=DNA%2CRNASeq%2FDRR024072&highlight= which attempts to include all public RNASeq studies for C. elegans. It has 54 columns that are mostly empty.

This issue proposes better support for such grids. The behaviour could be non-default and hidden behind a feature flag e.g. trackSelector.flexibleColumns [true, false], although if it works well I believe it will improve all grids.

1. Instead of setting grid width to be a certain amount of pixels, set it to "auto" here: https://github.com/GMOD/jbrowse/blob/544e3e9ee63804e4427ef14ca181af658c26e355/src/JBrowse/View/TrackList/Faceted.js#L383

2.Add new behaviour in _updateQuery, immediately after or before a call to this.dataGrid.updateQuery here: https://github.com/GMOD/jbrowse/blob/544e3e9ee63804e4427ef14ca181af658c26e355/src/JBrowse/View/TrackList/Faceted.js#L797 that controls visibility of columns.

Columns in Dojo can be set to visible or not via

this.dataGrid.layout.setColumnVisibility(<column index>, <true|false>)

We would iterate through columns, and for each column determine whether it has any content or not. I am not sure how yet, I hope this will be easy through this.trackDataStore.

I am a bit concerned that this could be a lot of DOM operations. On my grid, in its worst possible case of ~5000 rows and 54 columns, setting visibilty of columns to random values takes on average 1.15 seconds on Firefox and a modern laptop as measured through this benchmark:

var timestamps = []; for (var c = 0 ; c < 100 ; c++) { var t = new Date().getTime(); for(var i = 1; i < 54 ; i++ ) {this.dataGrid.layout.setColumnVisibility(i, Math.random() < 0.5)} ; timestamps.push(new Date().getTime() - t);} ;

I am happy to have a go at implementing this if maintainers think it's a desirable functionality and a good plan.

[wbazant]

If you happen to want to try exactly my installation of JBrowse, you could copy the url, replace http://test.parasite.wormbase.org with localhost:8080, and run Nginx with this config:

worker_processes  1;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       8080;
        server_name  localhost;
        location /jbrowse-data {
          proxy_pass http://test.parasite.wormbase.org/jbrowse-data;
        }
        location /jbrowse {
          location /jbrowse/dist {
             # ~/dev/jbrowse : where my copy of JBrowse is 
             # ~/dev/jbrowse/dist is where Webpack refreshes the bundles
             root ~/dev;
          }
          proxy_pass http://test.parasite.wormbase.org/jbrowse;
        }
    }
    include servers/*;
}

[cmdcolin]

I'm a bit concerned about this idea of "dynamically" hiding and re-showing columns.

I feel like, in some sense, the concept needs to be simplified. Dynamically hiding and adding columns indicates to me the grid model doesn't make sense.

What if it's just a simple "search" engine style thing where you have both browse and search, but it doesn't attempt to list things in grid format.

[cmdcolin]

I'm actually kind of concerned because if I'm not mistaken, even when you have 0 tracks selected on this website (http://test.parasite.wormbase.org/jbrowse/index.html?data=%2Fjbrowse-data%2Fcaenorhabditis_elegans_prjna13758%2Fdata&loc=X%3A1138508..1158445&tracks=&highlight=) the scrolling is still very slow. I am wondering what might be causing that and if it's somehow related to having a ton of metadata!

[cmdcolin]

The concern about large trackList metadata moved to #1163

[cmdcolin]

I sort of feel like this, as proposed, will probably not be incorporated. Note that TrackList code can be subclassed and implemented as a plugin (e.g. the trackSelector.type can be reference a plugin class in the config) so you could implement this in your instance :)

Note we are also brainstorming "future track list" ideas and if you have any new ideas let us know :) I might close this issue for now though

[wbazant]

@cmdcolin yeah it was just an idea. In the end I'm just showing much fewer columns for C. elegans, and I think it makes the thing usable enough. I'll let you know if something comes to mind!