allow users to fetch additional models (>100)

[harryhoch]

see https://github.com/monarch-initiative/monarch-app/issues/388

[yuanzhou]

@harryhoch just want to get your opinion on this and the adaptive re-sizing based on the container screen size that we talked about last week. Actually in the "analyze/phenotype" page, under search section, we can specify a user-defined limit for single species query. But it always only displays the first 10 (can be changed in code config) of each species in the cross comparison mode.

When you run Phenogrid as a standalone widget, or with 3rd-party embedding, there's no UI controls for users to specify how many models to fetch.

Actually now I'm thinking that the adaptive re-sizing has some relationships to this issue. For example, if the users are allowed to define 5000 models to show in the grid from the initial UI, even on a huge monitor's screen, we won't be able to show all the 5000 columns without scrollbars. So we'll show the horizontal scrollbar, and in this case, adaptive re-sizing will update the viewport and decide how many columns to show in the grid region, as well as updating the scrolling range automatically. I almost feel this user-defined number of columns needs to be addressed first, and the adaptive re-sizing of the viewport is just UI tweaks.

[harryhoch]

yes, understood, but the question is really this - can we use the size of the containing div element to figure out how many columns to grab?. If we can do that, we don't need a user control, do we?

[yuanzhou]

We sure can use the size of containing div to determine how many columns to grab. But this doesn't make more sense than a pre-defined number (the current cutoff number is 100, set by the returning JSON). Because no matter how many columns to display, we can always fit them into the containing div with the help of scrollbar. In short, the number of columns doesn't matter since we have the scrollbar.

The user control determines the size of dataset. And the size of containing div determines how many columns to show on viewport upon scrolling.

[harryhoch]

ok, fair enough. Can we add a configuration option?

[yuanzhou]

I'm debating with myself between these two approaches.

1 - Still load 100 (the default cutoff value) models for single species mode upfront to render the initial Phenogrid. Show a control in the OPTIONS dialog and allow the users to hit a button and add another 100 models at a time. No button to remove the already added models since they can always scroll back. It also simplifies the dataset update.

The downside of this approach is when the users fresh the page, it's back to the initial state.

2 - Add a configuration option in the constructor or global config file to specify the max number of models to grab before rendering the grid for the first time. This should be relatively easy to do since I've added the "limit" in search mode under "analyze phenotypes". And it's only for single species mode. But we can still load, say 1000 models, for each species and only render the first 10 of each for the cross comparison mode. Good thing is when we switch to single species mode, the data will show immediately. No extra data loading and dataset updates.

The downside is it may take a little longer time initially to load everything upfront.

[harryhoch]

@yuanzhou, we might just have to try it out. Can you try #2 and see how well it works?

[yuanzhou]

@harryhoch yes sir, I'll play with that. Thanks!

[yuanzhou]

Interesting findings. I tested with the index.html for single species since it has a long list of phenotypes. After I increased the limit to 1000, the data loading seemed fine, but dragging the minimap and the scrollbars caused a very noticeable delay on the updates of the shaded area as well as the scrollbar slider and grid cells. Still had this problem after I reduced the limit to 300 (a little better in delay). So I disabled the minimap and all of sudden the scrolling and grid update worked smoothly. I think the reason is when we have a matrix with a huge number of columns, we also created so many mini cell elements in the DOM, hence the DOM update gets very slow accordingly. I'll refactor the minimap creation to minimize the SVG elements that will be created. Ideally if we can create a minimap and show it as an image, then we won't insert that many DOM elements. Essentially we don't really update those mini cells. @harryhoch please also let me know if you have other ideas.

[harryhoch]

@yuanzhou, an image sounds great, thanks.