Web Viewer - Conceptual Designs - ROI Drawing

[gusferguson]

Looking at designs and mockups for introducing ROIs in the Web Viewer.

Particular ideas put forward so far:

• thumbnails down the side and next-previous buttons
• have all the information in main viewer right-hand pane available
• ROI drawing and management tools on palette
• maximise viewing area because of increasing standard resolution of .dvs etc.
• able to view metadata and change settings with ROI palette available

Not addressed now:

• any changes to the Measuring Tool/web equivalent layout/tools/icons to be done at the same time
• what to show and how to show other tabs on ROI palette/pane e.g. intensity view etc.
• rotation of image

Shows full viewer opened in new tab/new window. Thumbnails on left, next-previous buttons at bottom (could shift over to left more if desired) Right-hand pane as per main viewer with addition of ROIs tab (or whatever it is called) Settings pane selected.

Note: here and throughout - no scroll bars for image - will pan by click and drag only.

ROIs pane selected.

If decided to allow to open as palette - button top right hand corner of RHP

Clicking that opens ROI tab as re-sizable palette and collapses RHP down. Close palette button top right corner of palette - would revert to previous.

If desired RHP minus ROI tab can be expanded and re-hidden. This shows zoomed in image with z and t sliders visible over image

Comments and feedback welcome.

Will consider other details such as toolsets and icons in the meanwhile.

[joshmoore]

If I'm understanding the center panel here, we're in the context of a single image and the ROIs are being shown:

roi1*  |  [
roi2   |     full viewer
roi3   |     for roi 1
...    |                      ]

Is this the beginning of a more general center panel strategy? e.g. click on a dataset and get:

img1*  |  [
img2   |     full viewer
img3   |     for img 1
...    |                      ]

[gusferguson]

@joshmoore - this is not a centre pane strategy as such - it is a stand-alone viewer - I have just used some of the space maximisation that I had in the centre pane concepts. The viewer would be in the context of a single image, but the thumbnails would allow cycling through a dataset.

[joshmoore]

Understood. Thanks, @gusferguson.

[jrswedlow]

Thanks @gusferguson for taking this forward. A good start. I agree that we probably a new tab concept rather than a middle pane-- my bias is that there is too much functionality to include in a middle pane.

Two general comments:

1. I think we could do more on the left side of this to help use grok larger sets of images. Simple thumbnails are insufficient. Mouse over gives filename, other metadata? What about DigPath use case, where each image has several associated images?
2. OK, I will say it-- the ROI palette is way too tied to the insight metadata display. Can we liberate the web client to do what needs to be done, instead of being slave to insight? Currently, you have way too much stuff in the palette, and this will not scale; you'll need tabs in the palette, or some other device to make it manageable.

Let's meet and go over this, possibly with @will-moore, @aleksandra-tarkowska, @jburel, and @graemeball.

[gusferguson]

@jrswedlow - thanks for feedback. We will definitely be working on the palette display, just used the layout above as a placeholder to get a quick start and to be able to compare the space taken up by the palette vs right-hand pane version. If we do end up using accordion tabs in some way here then it would be better to base them on the ones in the main window so we don't have a proliferation of different styles. Further mockups of palette layout etc. to come.

[gusferguson]

Working on options for palette.

Suggestion from @will-moore to have colour, line and edit drop-downs in tool bar as in Figure.

Colour drop-down (as in Figure)

"More Colours..." could give options same as current Viewer colour picker.

Line width drop-down.

Edit menu drop down.

[jrswedlow]

@gusferguson Thanks. Re:accordion; how will these contend with 100, 1000, 10000 ROIs? It will always break at some point-- would be good to know where.

[gusferguson]

@jrswedlow - I am not hung up on accordion tabs or anything else at this stage - still very much trying to work out what we need to show so I can work on the how.

Working through the use cases, I am also starting to wonder if any user would actually have a reason to want to interact with more than X number of ROIs. If you have hundreds, then you would probably want to work with a subset - using folders/filters etc to home in on the subset.

I feel there must be a number tipping-point at which users would be unable to do anything much no matter how they are displayed. Exactly what it is will probably be more a functional/technical question then a UI display question.

[manics]

There's probably not much use in showing the details of too many ROIs in the metadata pane, but some sort of visualisation would be useful on the image. For example it isn't much use showing 100,000 individual ROIs on the image , but it might be useful to show where there are "clusters" of them, kind of like how on google maps you can sometimes have a big marker indicating there are several markers close together, and you either click or zoom in to see the individual ones.

[graemeball]

I agree with the gist of the 2 comments above -- i.e. good to get a feel for how many ROIs / roughly where ROIs are if there are a lot, but a bad idea to try to render them all fully until you zoom in. This will definitely be important for working with slide scanner data etc. (is it just rendering that bogs down for too many ROIs at the moment? or is client retrieval of the ROI objects already slow?)

[gusferguson]

Some general points about the new Viewer:

The technology change in the Viewer is intended to allow drawing and display of ROIs in theWeb.

As far as I am aware no decisions have been made about what other changes/improvements can/should be added, in terms of the resources available.

Tons of RFEs and requirements that have been around for years: https://docs.google.com/document/d/1TgyJqYodEaUoL_CzcjfzbJcnpVTR57ZH2iZF5ICsOJY/edit

Must make sure that trying to cater for specialist requirements doesn’t make it less usable for standard viewing and drawing workflows.

The blindingly obvious - we should try and not lock the viewer into a paradigm/technology that we get stuck with (e.g. Insight Measurement Tool) - keep things as flexible as possible so functionality can be added by others or ourselves further down the line.

On specific points:

a. What is actually needed from the viewer:

• suggest use cases probably fairly well covered by the ROI folders use cases:

IDR/CS - https://github.com/openmicroscopy/design/issues/7 analysis - https://github.com/openmicroscopy/design/issues/2 learning/digital pathology - https://github.com/openmicroscopy/design/issues/8

• the straight-forward browsing, drawing and viewing use case.

b. Right-hand pane/thumbnails

• if we want to show more just thumbnails for a “next=previous” type interaction then we are starting to circle back to the “use the centre pane" arguments - there is no point in the image viewer just replicating the main window/data viewer
• what do we show for thumbnails with well images?
• mouseover showing metadata will not work very well with lots of metadata e.g. HCS/IDR images

c. Grouping ROIs

• what do you use to group? folders/proximity/type/intensity - all of these
• ? heatmaps better
• how will these be presented?
• have to know what users want to achieve from these interactions before we can get anywhere down this road

[gusferguson]

A tweaked mockup of Viewer with palette showing ROI - just to include the new toolbar elements and some other missing elements including - show scalebar, show ROIs, image link, Z and T counts and moved Prev/Next buttons - before they get forgotten.

NB - accordion tabs are still just acting as placeholders.

[atarkowska]

For example it isn't much use showing 100,000 individual ROIs on the image , but it might be useful to show where there are "clusters" of them, kind of like how on google maps you can sometimes have a big marker indicating there are several markers close together, and you either click or zoom in to see the individual ones.

in terms of web vierwer based on openlayers clusters are possible http://openlayers.org/en/v3.5.0/examples/cluster.html

[gusferguson]

A last couple of mockups - still using accordion tabs as placeholder - arising out of points raised in the client meeting. From here on will focus on different options for the palette itself - as discussed in client meeting.

ROI count below thumbnails.

Collapsing palette to toolbar.

Image details (from General pane) in mouseover. Metadata in terms of HCS pytables could be included. Display could become an issue from a volume POV. Retaining clickable links in mouseover would be a problem - would have to become a pop-up for that.