Rois from omero2

[will-moore]

Previously opened as https://github.com/ome/omero-figure/pull/165 Ported to new code layout by squashing commits etc.

This allows loading of ROIs from OMERO to add to figure panels as new Shapes.

To test:

• In figure, pick a multi-plane image (also good to test with single plane image)
• In the Info tab, open the image with iViewer and add some ROIs (if not already on the image). At least 1 of each type of shape and including an ROI with many shapes (more than 10).
• Bonus: include rotated Ellipses (need to use Insight).
• Save ROIs, go back to OMERO.figure, and in the right tab 'ROIs' click "Draw".
• Test that drawing shapes manually works OK as before.
• Click "Load ROIs from OMERO" - (NB: this should be disabled for images with NO ROIs).
• Should see a list of ROIs / shapes, similar to view in webclient with ROIs expandable to show Shapes.
• Expanding ROIs with many shapes will not render image for more than 10 at a time (to avoid loading too many image planes at once).
• Hovering over each Shape row displays it on the image. Clicking on row sets the Z & T of the image to the Z & T of that shape. Clicking "Add" will add the Shape to the image.
• Add Arrow, Line, Ellipse and Rectangle. Check they look same as in iViewer.
• Click OK to close dialog and save shapes to image panel.
• Open the "Draw" dialog for a different image to check the dialog is re-rendered correctly.

Follow up ideas:

• Maybe option to filter shapes by current plane?
• When Ellipses are added, x, y, rx, ry are recalculated from handles (to handle transforms) but this means that they are not detected as duplicates
• Maybe add Z & T sliders?
• Maybe show shapes on viewer when mouseover area!? - click to add?!
• Do we allow shape to be added if previously added & moved? Don't want duplicate IDs!

[waxenegger]

Great stuff @will-moore. I like the drawing and rotation abilities.

That seems to all work as you described it, tried a rotated ellipse as well.

After changing stroke width and color, I wanted more and change the fill color. Maybe another follow-up idea...

[will-moore]

To test the upgrade of Shapes json format from 5.2.x format to 5.3.x (ellipse.cx -> ellipse.x etc) we need to:

• create some ellipses in a new figure without this PR.
• File > export as JSON, check that "version": 1
• Maybe take a screenshot and/or make a copy of JSON.
• Save the file.

Then I will Open this PR again...

• With the PR merged, open the file from above
• Check that ellipses are still shown as before
• Check the JSON for "version":2
• Ellipses in JSON should now be like {"type":"Ellipse","x":115.1,"y":119.1,"radiusX":82.8,"radiusY":41.4,"rotation":50.3 ...}

[bramalingam]

Created figure without this PR on eel.merge, http://web-dev-merge.openmicroscopy.org/figure/file/17602 (user-1)

Lots of ellipses and I have a copy of the JSON file as well : yes it is "version":1:

{"version":1,"panels":[{"labels":[],"height":94.18951612903223,"channels":....

[will-moore]

After upgrade, we also need to test that figure export script works, as TIFF and PDF. (will need to manually upload the script).

[jburel]

• Description needs updated. arrows are correct now in iviewer
• Shapes like rectangle have a fill color, this ignores in figure. This could be used to "mask" areas so might be useful to have the option to turn on/off
• Listing the shapes that I cannot add clutters the right-hand size for no reason. I will not display them by default
• When added to the main image, it is annoying to have to go back to the Drawing window just to move the shapes. This is even more obvious if I click on copy/paste, the count increases but I cannot do anything. This part of the workflow needs review

[will-moore]

@jburel

• I've never supported "fill" of shapes in figure, mostly because I've never seen a figure with filled shapes. I guess we could add this sometime, but its not a high priority feature at the moment.
• Listing shapes you cannot add was what @pwalczysko and I decided in https://github.com/ome/omero-figure/pull/165#issuecomment-241434281. It's kinda like a disabled option in a menu. It shows you that the option/shape is not "missing", it's just not supported.
• Moving shapes on an Image when in the full figure view is not really viable since we use selection and drag in that view to move the image panels. It's like in webclient/Insight: you can't adjust rendering or ROIs on thumbnails - you have to do it in a full viewer (but you can copy and paste rendering settings between thumbnails).

[jburel]

• For the shape listing. In that case we should list the supported ones at the top. I had a large number of points and 2 rectangles and I had to scroll down since the rectangles were not visible
• I don't think the argument Copy/paste is valid. Since the copied shapes overlap the existing shapes and user cannot manipulate it. This is not practical at all

Let's discuss

[will-moore]

After discussion with @jburel, TODOs:

• Change "Draw" button to "Edit" in right panel
• Add extra text to the top of dialog (& rename "Draw.." to "Edit..."
• Move "Load ROIs from OMERO" button to toolbar and rename "Load ROIs" with longer text in tooltip.
• This button should be disabled when clicked (to avoid repeat loading). Could also be used to refresh ROIs (maybe in a future PR)
• Don't show preview 'thumbnail' for each Shape
• If 'Adding' a shape that is already there, offset the new one (same as when pasting duplicates).
• For detecting duplicate Ellipses, compare coordinates and assume duplicate if very similar.

[bramalingam]

Tried exporting, http://web-dev-merge.openmicroscopy.org/figure/file/17602

A figure file which was created before this PR was opened. https://github.com/ome/omero-figure/pull/190#issuecomment-281746124

And got the following exceptions when I tried to export as pdf,

No handlers could be found for logger "figure_to_pdf" /usr/lib64/python2.6/site-packages/reportlab/lib/utils.py:653: DeprecationWarning: tostring() is deprecated. Please call tobytes() instead. self._data = im.tostring() Traceback (most recent call last): File "./script", line 1665, in run_script() File "./script", line 1651, in run_script file_annotation = export_figure(conn, script_params) File "./script", line 1609, in export_figure return fig_export.build_figure() File "./script", line 690, in build_figure self.add_panels_to_page(panels_json, image_ids, page) File "./script", line 1311, in add_panels_to_page self.add_rois(panel, page) # This does nothing for TIFF export File "./script", line 823, in add_rois self.page_height) File "./script", line 130, in init self.draw_ellipse(shape) File "./script", line 319, in draw_ellipse c = self.panel_to_page_coords(shape['x'], shape['y']) KeyError: 'x'

And for conversion to tiff,

No handlers could be found for logger "figure_to_pdf" Traceback (most recent call last): File "./script", line 1665, in run_script() File "./script", line 1651, in run_script file_annotation = export_figure(conn, script_params) File "./script", line 1609, in export_figure return fig_export.build_figure() File "./script", line 690, in build_figure self.add_panels_to_page(panels_json, image_ids, page) File "./script", line 1307, in add_panels_to_page image, pil_img = self.draw_panel(panel, page, i) File "./script", line 1140, in draw_panel self.paste_image(pil_img, img_name, panel, page, dpi) File "./script", line 1503, in paste_image ShapeToPilExport(pil_img, panel, crop) File "./script", line 381, in init self.draw_ellipse(shape) File "./script", line 518, in draw_ellipse ctr = self.get_panel_coords(shape['x'], shape['y']) KeyError: 'x'

[bramalingam]

Same errors, looks like an issue with getting the panel coordinates! Is this related to/caused by my width and height PR (we handled the page-wise x,y coordinate management in that PR)? https://github.com/ome/omero-figure/pull/169

[jburel]

see also https://trello.com/c/zGuNePnu/4-load-rois-from-omero

[will-moore]

@bramalingam I'm trying to understand the bug you found. The Ellipses that are giving you errors on the figure seem to come from OMERO (ID matches an Ellipse on that image in OMERO). Can you remember how you created that figure, since if the figure was created without this PR merged, you shouldn't have been able to load ROIs from OMERO. Maybe that came from a previously created figure?

{"rotation":60.65125478487214,"strokeWidth":1,"type":"Ellipse","strokeColor":"#c3c4c4","id":1751}

I just tried this:

• With this PR in, I created a figure, added an Ellipse from OMERO (version 2).
• Without this PR in, I opened that figure and Saved it (version 1).
• With this PR in, I opened the saved figure (updated itself to version 2) but the Ellipse had lost it's x, y, radiusX and radiusY as above.

Does this sound feasible?

[will-moore]

To try reproduce the upgrade issue above, closing this PR so we can create figures without it merged.

[will-moore]

Recent changes to test:

• Change "Draw" button to "Edit" in right panel
• Add extra text to the top of "Edit ROIs" dialog "Draw and Edit..." etc
• Move "Load ROIs from OMERO" button to toolbar and rename "Load ROIs" with longer text in tooltip.
• Check button is disabled with suitable tooltip when image has no ROIs.
• When clicked, button is disabled while loading then updates to "Refresh ROIs" to allow re-loading.
• Thumbnail previews for shapes in the Shapes table have been removed.
• If 'Adding' a shape that is already there (click 'Add' repeatedly), offset the new one (same as when pasting duplicates).
• This should work for all supported shapes (line/arrow, rectangle, ellipse).

[waxenegger]

Checked the most recent changes and they all work.

[will-moore]

@bramalingam Can you try testing the upgrade of shapes again? @jburel Do you want to check that workflow is behaving as we discussed above?

[bramalingam]

Rechecked the following scenario,

1) Created Figure with ROI's without this PR. ( json should contain "version 1") 2) fetched this branch and opened the same Figure, and exported as pdf.

Can no longer see the exception seen in : https://github.com/ome/omero-figure/pull/190#issuecomment-284007902

and can see version 2 in the json as well.

[jburel]

Looking at http://web-dev-merge.openmicroscopy.org/figure/file/17602/

• Select the image in the top left corner
• Go to the label tab
• Click Edit
• In the ROI dialog.
  • If I mouse over a shape an "independent shape" The shape is added then removed from the display when I move the mouse away
  • If I mouse over a shape which is part of a roi with multiple shapes. The shape is added (if on the correct plane) but it is not removed from the display when I move the mouse away

[jburel]

RFE: A "undo" the copy paste for example will be nice (not in this PR)

[jburel]

Open the "ROI dialog", the tooltip of the Load ROIs button always indicates the number of rois of the first image I opened I will remove the number from the tooltip.

[will-moore]

@jburel - I can't reproduce the tooltip bug https://github.com/ome/omero-figure/pull/190#issuecomment-289783627

[will-moore]

As discussed with @bramalingam, when you "OK" the ROI dialog, the Z and T position of the dialog get saved to your figure. If you don't want to change the Z/T, then you can 'revert' to the original Z/T using new buttons:

[bramalingam]

The above functionality works as expected.

2 separate points to consider: 1) Display of the ROI/Shape ids from OMERO, and 2) Size/width of the rows/columns in the ROI table.

Also, an edge case workflow where an ellipse ROI/shape has its centre outside the bounds of the viewport, throws a not so informative dialog box. Maybe update the text in the dialog box or handle ellipse's better when there are near the edges of the image? (Can be ignored as well, because in a user workflow I wouldn't expect the ROI's around the edges to be useful in many ways)

[will-moore]

Added IDs to the ROI/Shapes table to look more like iViewer cc @waxenegger

[waxenegger]

Latest commits work as as intended. Shape Id is showing and tooltip gives me coordinates.

[jburel]

similar to iviewer: I am not sure about the shape/roi ID. Especially with the high number

[jburel]

Few comments:

• Remove the id of ROI/shapes from table. Move to tooltip
• Improve message when trying to add a shape to a zoomed image.

[jburel]

@will-moore I cannot reproduce the roi count issue, I don't think it was ever a problem