Dataset viewer redesign

[LachlanStuart]

Figure out how the Annotations page should look to support other use-cases & upcoming features

[LachlanStuart]

Ideas:

Overall design:

• Design the page around viewing a single dataset at a time (instead of allowing filters to select multiple / all datasets) so that we're able to show dataset-specific information such as marked regions, pathway enrichment, PCA/TIC/etc. diagnostic plots
• Tabbed design probably best - currently we only have an annotation-centric view, but ideally users' first view of the dataset would be an overview, which would allow them to drill down to the details through different paths.
• Can we drop the one-DB-at-a-time limitation?

"Overview"

• Show metadata, external links (DOIs, etc.), full list of contributors
  • Make this editable by the submitter?
• ~Per-database counts of annotations~
• "Gallery" tile allowing the user to flip through diagnostic images:
  • ~High-res "Ion Thumbnail" (in future this could link to the ion-image overlay tool with the ion thumbnail annotations pre-selected)~
  • ~TIC and Lipid-TIC~
  • Defined regions
  • ~Per-pixel average m/z shift~
  • ~Per-pixel most abundant ion~
• ~"Compare with" widget~
• ~Actions: Download, reprocess, share~

"Details"

• Tabs for "Pathways", "Molecules", "Formulas", "Ions". Each (except "Ions") would have grouped rows that expand to show all ions that contribute to the grouping
• Some mechanism for selecting multiple ions, so that their ion images can be merged into one, or compared side-by-side
  • After selection has been made, show a button to go to a custom viewer where each annotation can be given a color, scaled, etc.
• When a set of regions is selected, annotations table expands to include one column per region indicating some summary statistic (selectable from mean/median/min/max/total intensity or % occupancy)

"Ion image viewer"

• Fullscreen button, so that people can get a bigger image for saving
• Region selection tool:
  • Polygon, rectangle and variable-width brush tools, each with both additive and subtractive modes

[LachlanStuart]

Notes from Måns' demo of MSiReader

Comparing DSs:

• Loaded a folder of files and selected "tile [2] x [2]" as the layout
• Re: "Do you prefer filtering by '% of spectra containing peak', or relative intensity"

  "Most of the time I only care about things that are e.g. 5x more abundant in one of the regions"

Overlaying ion images

• MSiReader has customizable "gain" for each ion image
  • Re: "doesn't that make the images incomparable?"

    "We're always more interested in spatial distribution than specific intensities"

• Sometimes useful to sum multiple images (e.g. for a family of lipids)

Diagnostic plots:

• Showing the TIC image helps to check comparability of datasets, highlights any significant errors in the dataset acquisition
• The plot where each pixel is colored by the m/z of most intense peak per pixel also helps to show if datasets are comparable & how consistent the data is
• Also showed plotting a line of pixels between two drawn points, showing their intensities as a bar chart.
  • This is useful for comparing actual intensities, because heatmaps lie to you due to human visual perception
  • This can also be used for measuring leakage, because it allows you to count how many pixels the metabolite has spread outside of the region where it should be localized

Intensity normalization

• Thinks TIC normalization probably shouldn't on by default
• Normalize to uploaded image could be useful (e.g. a previously generated summed image)

  "Normalizing to the sum of lipids can be better than TIC, as TIC is much more sensitive to experimental errors"

General UX

• Doesn't like the inline filter buttons in annotations table, as they're too easy to accidentally click

[richardgoater]

@aeisenbarth and I had a brief discussion about general UX at the weekend. We echoed Måns' point about the inline filter buttons, and also mentioned some unexpected pop-ups in certain places. @aeisenbarth would you like to comment?

[LachlanStuart]

@richardgoater Were the unexpected pop-ups from the candidate molecules popover?

We've reduced the vertical size substantially by limiting the number of items it shows, but it still takes up a significant amount of horizontal space, especially with the isobars warning or long molecule names. Here's a particularly bad example:

[aeisenbarth]

• Blue filter button that appears on hover:
  In some columns it is right-aligned, in the annotation column it is left aligned after the text. The issue is that when clicking a row to select an annotation, the button might surprisingly appear under the cursor and filter instead only this annotation instead of selecting it. For consistency we could consider aligning the filter button and pop-up to the right.

• Pop-up "Candidate molecules":
  For a task where you are not interested in that information, when it appears you have to move the mouse quite some distance and wait to make it go away. For the task of scanning the sum formulars for interesting molecule names, it is understandably useful to have that information accessible with little effort (hover instead of clicking). Could it be put into a fixed section on right panel? Or would a dedicated molecular class browser (in the molecule tab) be more suitable for that task and make the pop-up obsolete?

• I am sometimes interested in the pixel coordinates for comparing positions of features (but this might be specific for debugging SpaceM). The tooltip on the ion image shows only the intensity value of the pixel under the cursor.

[richardgoater]

Collected Slack Comments

• not being able to browse through molecules with the keyboard without clicking in the annotation table (if I clicked anywhere else before)

• lack of copy-paste of molecular formula (which I often use to search elsewhere)

• not being able to see/copy the full dataset name for a long name (the only option to start adding a new dataset, then click-copy within the field, super weird)

• the optical overlay page does not shrink images to fit my screen, so I have to scroll up and down while aligning (and frequently requiring the MS image to partially overlap the UI elements so that I have to tab around to press a button). Dream: let me anchor a known point and just morph from there.

• Can't look at mass spectra.

• that tooltips with the names of molecules (for many isomers) block the whole screen when hovering over a molecular formula

• You have to wait for files to complete uploading before you can click the submit button. Make it active when files uploading starts. On submit, wait for the files uploading to finish before actually submitting

• No "back" button overall, but the most annoying when selecting a molecule (either by clicking) or by copy-pasting it into the field. Right now I’m often open another tab to “preserve” the view I need

• no auto submit button after upload

• no ability to go back to choose file to upload if something fails. Only drag and drop works or you need to refresh losing input data. (x2)

• remembering the last input field as opposed to last submitted field for metadata after page refresh

• slightly different annotations dont show up in search.. ie results submitted for matrix dan and 1'5 diamino... would have to be searched separately.

• “Select” repeats in almost all filters of the “Add filter” field

• you can't sort datasets by date

[LachlanStuart]

Notes from the meeting

“Views” of the data

• Lists:
  • Ions (current Annotations page)
  • Molecules
  • Pathway enrichment
• Chemical-space plots:
  • UMAP of morgen fingerprints for entire DB
  • m/z vs “cLogP Polarity”
• Tag statistics / filter (with view of counts of hits per tag)
• Find similar datasets
• Viewing spectra (specific pixels, sum spectra, regional sum spectra)

Filters:

• By some sort of tag from mol DB
• By user-selected tag

CR: “I would never not have the Molecules section open”

Structure:

• Dataset overview
• Metadata
• Gallery
• Widget for comparison
• Links to other views
• Comparison overview
• Links to other views
• Region definition tool (combine with optical image alignment tool?)
• Annotation page
  • shows 1 annotation + all metadata

Further discussion: These as separate pages:

• (UMAP/pathway list/mol class list/region list) -> Annotation list
• Annotation table -> Ion Image

Or somehow combine it into one page, or make the UMAP/etc a selectable right-side-of-the-screen fixture?

[LachlanStuart]

From today's strategic meeting - @LachlanStuart responsible for breaking this into tasks, analyzing what needs further analysis, triage, back-end dev, front-end dev.

[LachlanStuart]

I'm going to move this off the METASPACE v1.7/1.8 board on to the Roadmap board. This task doesn't warrant closing as it's not really complete and still has a lot of good info in the comments that may be used for reference later, but work is continuing in other, smaller tasks (currently #575 and #561), so something must be done with this task to stop it cluttering the main board.

[LachlanStuart]

I'm assigning @theodev to triage this (close, or break into concrete tasks for implementation) as he has the most context for it.