suggestions for future work on `alpha-rarefaction` visualizer

[jairideout]

Improvement Description The new alpha-rarefaction visualizer @maxvonhippel authored is almost ready for merge (#128) and will be included in the 2017.8 release! :tada:

When reviewing #128 I had some suggestions for future work on the visualizer, but they are outside the scope of #128 so I'm posting them here. We can split this issue into smaller issues whenever we get around to working on these changes.

Proposed Behavior

• Allow user to supply multiple metrics (default behavior computes 2-3 metrics but users can only specify a single metric if not using the defaults). Done: https://github.com/qiime2/q2-diversity/pull/158
• Hide the secondary plot (number of samples vs sequencing depth) when metadata isn't supplied. Since there's no grouping in that case the plot will always be y = 1.
• Don't display negative x-axis bounds in either plot (currently it's possible if min_depth is close to zero).
• Add a button in the legend to "deselect all" (it wasn't clear that the "select all" button could be used to deselect all).
• Add some help text in the visualization describing how to interact with the plots, legend controls, etc.
• I'm always seeing a scrollbar on the right side of the plots even when there's nothing to scroll over. Can the scrollbar be displayed only when one is needed? I'm running Chrome on Ubuntu.
• Option to export plots as SVG
• ~Add error bars~ Now #179.
• When metadata is provided, add a UI element to disable grouping by metadata category. That way, users can get a per-sample (ungrouped) plot without having to regenerate the visualization sans metadata.
• When metadata is provided, add an option to color lines by the metadata group they belong to, but avoid grouping the per-sample lines.
• Support hovering on data points in the line plots (when a line is selected) to see the x, y values (in a tooltip or some other display).
• Reorient the CSV export format to have columns sample-id, depth, iteration, and value. This structure avoids dynamically named columns, and grows row-wise instead of growing both row-wise and column-wise. This new format won't be loadable as qiime2 Metadata because the sample IDs will be repeated (due to the hierarchical nature of the index: sample-id > depth > iter), but it'll be easier to plot in other programs and has a predictable structure. This new data layout will simplify a lot of the pandas-munging code on the Python side of things too (it's pretty complicated right now due to MultiIndex use, and pandas might not be the right tool for the job anyways).

References

1. 128

2. https://github.com/qiime2/q2-diversity/pull/158

3. 179

[jairideout]

These suggestions also came up on the forum.

[antgonza]

Thanks for putting this together. One more suggestion, store intermediary files (rarefied tables and their alpha div vectors).

[maxvonhippel]

I like potentially providing the option to store intermediary files, though for performance reasons I would prefer not to have it be the default. The "deselect all" idea, scrollbar, and help text could make for a good/easy/fast joint PR, IMO. RE The first point about providing multiple metrics - I think that we will/can allow this but we had to wait for Set to be fully supported in the q2cli and in Provenance ... am I right that this was the hang-up, @ebolyen ?

[ebolyen]

Re: intermediary files. That falls in line with turning this action into a pipeline (which is now supported, and planned for the near future).

And for multiple metrics, that is now supported as of 2017.10. (although sets still look a bit ugly in provenance, but they are accurate according to yaml spec)

[jairideout]

Grouping the intermediate file comments:

Thanks for putting this together. One more suggestion, store intermediary files (rarefied tables and their alpha div vectors).

I like potentially providing the option to store intermediary files, though for performance reasons I would prefer not to have it be the default.

Re: intermediary files. That falls in line with turning this action into a pipeline (which is now supported, and planned for the near future).

:+1: for an option to store intermediate files. This feature can't be implemented right now because the QIIME 2 framework doesn't support variadic outputs, so there's a missing piece even though Pipelines are now supported.

@maxvonhippel I'll update the issue to indicate which bullet points have been completed (e.g. the work @jakereps did with Set support and multiple metrics).

[gregcaporaso]

The recommendation of starting with the median frequency per sample as max-depth seems pretty reasonable based on a few studies that I've been working on. It might be worth making this a default, and including recommendations on when to change it in the associated documentation. Otherwise we can expect this to come up as a question a lot (eg).

[nbokulich]

another suggestion from the forum:

allow user-defined coloring

[maxvonhippel]

@nbokulich that makes sense as a feature since it reflects colorization options in other plugins such as Emperor.

That said, the design might not be completely straightforward. Do we provide a list of palette options? Do we allow the user to upload some sort of palette file? (Is there a standard format for palette files, which could map deterministically to specific values in the plot?) Maybe these questions have all already been answered in the development of some other QIIME 2 code; I'm not sure.

[nbokulich]

@maxvonhippel emperor could be a model for this (i.e., support interactive palettes).

However, I am naive to how the plots are being generated and if that requires fundamental changes (e.g., switching to something like d3), it is probably not worth it. For users who need such a feature to generate publication-quality figures, manual color changing would be equally straightforward in Adobe Illustrator or equivalent free image editing software.

[maxvonhippel]

@nbokulich in this visualizer we are already generating the plots with d3. That said, I agree about manual color changing, especially with the .svg files.

[jairideout]

forum x-ref

That forum topic pertains to this bullet point:

When metadata is provided, add a UI element to disable grouping by metadata category. That way, users can get a per-sample (ungrouped) plot without having to regenerate the visualization sans metadata.

[ebolyen]

forum x-ref re: export option

[thermokarst]

This recently came up on the forum (re: export option)