Open gusferguson opened 8 years ago
Even before I read your "indication of ROI distribution along Z and T sliders (still to come)" I thought it would be a great feature - like Chrome does when you "find" a word and it's distribution is shown in the slider:
This could be enhanced to highlight shapes that are in the same ROI as the currently select ROI, as well as indicate ALL shapes with a less bright colour?
One thing to consider is how well we support multiple shapes on the same Z/T plane within an ROI. Insight doesn't support it at all (crashes). We need to at least allow it when we find this in the DB, but do we allow users to create multiple shapes on a plane for a single ROI?
I don't think we ever need to show a Line profile AND a Histogram. Line profile for lines, histogram for shapes.
Clone tool is nice idea. It needs to be beside other shapes in the toolbar, since it is a mutually exclusive / alternative tool to other shape creation buttons.
Reminder: also need some way to distinguish non-attached "overlay" shapes from attached ones.
Some features of Google maps might be useful to consider - E.g. organisation of multiple shapes into "layers", editing shapes, floating palette & toolbar etc
@gusferguson There's alot to like here. @will-moore 's slide idea would be GREAT!!! Finding ROIs in large stacks or timelapse sequences is sometimes quite hard. I suspect layers, pallettes, etc need to be considered in the context of large (>50) ROIs/image. GMaps sometimes frustrates me with its insistence on placing layers etc on the stuff I want to look at.
Keep 'em coming!
I like your ideas @gusferguson. Viewing it from the user's perspective I believe that lots of what you mocked up is quite useful.
@will-moore in general there are quite a few products out there that do one or the other thing well, particularly if one looks at the features. there are, however, a few things to be said:
Don't get me wrong I'm not trying to be exclusive or negative, I just wanted to point toward some pitfalls that's all.
I have one other nitpicky (technical) remark in regards to @gusferguson's z/t orphans: I'm sure it could be implemented easily enough in the front-end (I do have a context menu on the viewer that I'm working on so that one could assign regions/clusters of regions to a specific z/t easily, and putting a list of orphans on any layer is also not rocket science) but, in the end, a corresponding decision has to be made how orphaned shapes are to be persisted and dealt with in the data model. Does one need another table since there is now a n:m relationship? Or does one simply store them redundantly for each t/z anyhow? Bear in mind that that impacts on both the storage effort as well as the use thereafter. One is at least in the first instance a shared shape (therefore less storage) although the user could later decide to just edit that one shape in that z/t context so that it would have to be singled out and stored individually anyhow. Once again, no showstopper by far, just something to keep in mind.
ok, I have finished my comment of epic dimensions. ... ;-)
I have actually a question since I'm at the moment trying to find the best way to interact with regions and there are a few options available. In specific I'm talking about selection and selection if one has a clustered view, something that will be (at a minimum in the web) a necessity IF the goal is to be able to view a large number of rois, say more than 500 only actually, in what is a small viewing extent of the image. The view can get cluttered and overloaded quite quickly, but depending on the overall image size, even for larger images (beyond 10,000 pixels square let's say), anything over 10K shapes in an zoom level that reveals the entire extent can be a bit of a challenge and clustering seems to be a not-too-shabby solution.
But how does one best interact with the clusters. Does one reveal what's underneath on mouse over? I tested that and it works well in webkit based browsers but not well in Firefox or IE. Does one reveal them when clicking on them (via context menus rightclick or left simply)? Or should the user just zoom in onto an area before they get to see in which case one would have to have knowledge of what's there or otherwise it's a bit frustrating to find something I think...
What should one do with multiple selections when in a clustered view? If I reveal too much I'm sort of back in the unclustered view because all of the sudden I want to show lots of things again, so perhaps only one cluster and how many shapes should be the limit that are behind that cluster?
And, even worse, what would I do if I transition back/forth between a zoom level that needs the clustered view and one that shows the individual shapes (I'm zoomed in already). If I select something there, should it be still selected when I'm zoomed out and back in the clustered view. Should I make it sticky, and if so, what happens if it is sticky (selected) and I then work in another region and perform an action on the selected things and forgot that somewhere else I had selected a shape ...
Honestly, I don't know how to deal with those questions and they have technical implications as well.
@waxenegger Apologies for the confusion. I wasn't suggesting we use google maps at-all, except for getting ideas/inspiration from their UI (hence the screenshot).
@waxenegger In the current roidisplay.js in the webclient, to find what shapes to show for the current plane we simply iterate through all shapes and pick those with matching Z & T OR no Z / T set. There's no duplication of shapes so our js model matches the OMERO model.
@waxenegger For clustering, it would be good to see your current progress on this. I can imagine problems when clustering shapes of different type/size. Clustering works well on maps for points, but if you try to cluster some points with lines / ellipses / rectangles that could be as large as the image itself then how does this work? I can see that clustering will help prevent the UI from becoming too crowded, but with a large number of shapes, we will still hit bottlenecks with retrieving and marshalling all those shapes into json. It may be that we need some pagination strategy, with UI widgets for loading / browsing an overview of the shapes. Possibly with the server providing a summary / cluster of the ROIs and Shapes rather than loading ALL the data to the browser. These issues are likely to have a big effect on any client-side data model so are important if we want to handle a lot of data. Q: do we have an idea of the numbers of ROIs/Shapes we are intending to support? cc @sbesson @manics. I feel that clustering of shapes on the image viewer itself may be a useful solution in some cases but is not going to be the main blocker to working with large amounts of ROIs and Shapes. It might be that there's some threshold number of Shapes where we no-longer allow editing in the client (E.g. created by analysis results) so we may not need to handle clustering and shape editing at the same time?
@will-moore
about google-maps: I wasn't quite sure. either way it was just a remark and by no means criticism
regarding the z/t shapes: this would be only a potential question for the implementation of the z/t "orphans" that were mentioned. Unless they are purely temporary in a sense that they are not persisted the question does not arise. If they are stored as not belonging to a partiular z/t, I suppose they can stay without designated z/t as well and there is no issue. Same is true for backend/data model and front end. But once they start belonging to something and are being treated unique they need to be stored as such. Yes, duplication, is therefore not a good term at all I agree, only if one were to store a db record for such an occasion to begin with.
@will-moore
thanks for input on the clustering. yes I aim at showing some next week, tonight I'll have to go in a minute, sorry. I'm not sure I understand the size of the shape comment because any shape by itself could potentially cover the image if drawn big enough and filled without transparency.
A common approach is to just place a circle in kind of the center of the group of shapes as a representative and, on demand, you show them. This does help a bit with performance and sort of hides the complexity of the scene.
What you say about retrieval of data is true and I did mention in one of the meetings with Chris that selective querying on demand in terms of z/t could help if one were to have a lot of shapes AND not all in the same plane. If they are all in the plane the transfer would take a while and one could roughly calculate the number of bytes by estimating the size of the average json definition for a region and then assess how big the content would be and what is feasible and what not. But say one region definition needed to render in the client is 1,000 bytes (I think that's generous, right) then 10,000 would mean 10MB which takes a while, no doubt, but it's doable.
Yes there is also the marshalling that needs to be done server side, I didn't forget about that, and I suppose that needs testing.
These questions remain, but strictly speaking, they are the same for say the mask. At the moment one could generate a mask about 1GB large (db column limit) and if one were to retrieve it, it would also be a bit of a strain on the server and the communication, and if one really did it, one would have to resort to other strategies.
I believe there is also some room for improvement to make the regions work up to a certain threshold and although I don't have a concrete number I'm pretty sure it's high enough to need something in the client because the number might overwhelm the viewing extent.
@waxenegger Re clustering - using this as an example: http://openlayers.org/en/v3.15.1/examples/cluster.html All the features are points, but what if one of the features was a road that went from one side of Africa to the other? Would I be able to see the road when fully zoomed out or would it be clustered with other points? How much would I have to zoom in to see it? If I have to zoom in until every feature is not clustered then I wouldn't be able to see the whole road at once?
Thanks for all the feedback - I will press on with the remaining mockups and work on the suggestions. @waxenegger - re ROIs not attached to Z or T - that is supported in the model already, just not exposed in the clients.
Further mockups in - Web Viewer - Brainstorm - Concepts on New Interaction Approaches - 2 https://github.com/openmicroscopy/design/issues/34
Summary of points raised in this issue - Viewer Brainstorming 1:
ROI Distribution wrt Z and T
Will:
Enhancements:
Jason:
Graph Display:
Will:
Rendering Settings
Clone tool:
Will:
Overlay Shapes (shapes not attached to any Z or T)
Note: these are already supported in the model
Gus:
Layers
Will:
Harald: Some caveats:
Clustering
Harald: Will be needed if:
500 or so ROIs in a small area of the image
10K shapes in a 10Kx10K pixel image Questions with interaction with clustered images ? do you:
Will:
Harald:
Will:
adding the following for ref https://trello.com/c/yuRPX18c/65-viewer-issues
Having talked over the problems with legacy and design in the existing Insight Measurement Tool with J-M, we had some new ideas on how to accommodate ROI workflows better in the upcoming Web Viewer.
This is a first blast at trying to formulate the sorts of UI interactions that might make some of these ideas possible.
This will be expanded as more of the functionality and interactions start coming together and I do mockups of them.
Please add comments and feedback.
Caveats:
The key points raised and this brainstorming aims to address are:
Requirements/Task analysis
This still needs to be expanded with lab users.
Taking an abstracted view, the diagram below gives an idea of the scope of the mockups.
Mockups:
Default Viewer on opening:
Any of the RH pane tabs can be broken out as palettes allowing more flexibility
This is normal 512x512 image with RH pane open in ROIs tab
A couple of new toolbar items
Showing the drawing tools from the ROI tab as a palette
The various propagation options can be selected and then the Drawing Mode can be collapsed to maximise real-estate. The idea is that you get into your mode of working and then add ROIs as efficiently as possible by:
Improving the dynamic interaction between the analysis and drawing
This could be augmented with dynamic Intensity results while drawing
These could both be dragged out as palettes
At any stage full results can be dumped out:
Another advantage of having both RH pane and palettes
And last but not least
To be continued ...