segmented object exploration and annotation

[tischi]

Hi @ssgpers,

I wanted to implement an imagej plugin for "segmented objects exploration and annotation" and it would be great to have your input. I think I have all of the code (even already in production mode in different contexts), but I wanted to have a fresh look at it and see whether I can make a generic reusable version.

Plugin name

Is "Segmented Objects Exploration and Annotation" a good name?

Input

I was wondering about the requirements for the input data. I was thinking about the following:

• Table where each row corresponds to one object (required)
• Table columns
  • Relative path to label mask image (required)
  • Object label index (required)
    • The index must match the index in the label mask image
  • Object coordinates (required)
    • Typically the centroid
    • This is needed to zoom to an object, e.g., when an object is selected in the table, or when going through objects one-by-one during annotation or review
  • Relative path(s) to additional (intensity) images (optional)
• Label mask images as indicated in the table (required)
• Optional images as indicated in the table (optional)

What do you think?

[tischi]

Other questions came to my mind (which already popped up on other occasions):

1. Is a table allowed to contain multiple label mask images?
2. If yes, what defines objects and images that are in the same table? Should we put some restrictions on this?
   • What I would really like to require is that all images (and objects) that are in the same table could be presented in the same coordinate system. I think this would be really nice in terms of, e.g., saying that one table corresponds to one plate of a HTM experiment, or one coverslip of an adaptive feedback experiment.
     • This would require an additional column in the table which, for each image-set (label mask and intensity images) also specifies the location of the image.
     • A challenging question would be in which coordinate system the object coordinates would be given? In the global coordinate system or the local one of the image in which they live?

[ssgpers]

Sounds very nice idea if this can be more generic.

Probably most input from my side will be after I try to put it to couple of projects and see what is missing.

For now few comments:

• Name is good, but shorter version for advertising would be useful. One idea (not so good): "Mask explorer and annotator".
• I think intensity image should not be optional,rather obligatory. If you think there are causes without raw data, user should be able to select mask image column as row image column.
• Can 3D masks and objects be covered in your paradigm?
• Multiple label mask images - rather yes, for example nuclei, cells and spots within cells. For annotation purposes user should select which mask to work with.
• For coordinate system question I currently do not see optimal global solution provided variability of projects I deal with. I see the point for classical high throughput screens, but even if I take example of live experiment, should be the time to be another coordinate for example. And for feedback micros copy global coordinate values might or might not make sense. I guess this question we need to discuss in person.
• In any case for the beginning I would do the following requirements (maybe obvious anyway):
  • Mask and intensity images have the same format (number of pixels) in 3D
  • Object positions are specified in pixel coordinates of the mask image.

[tischi]

1. 3D works in my framework
2. Mask and intensity images have the same format (number of pixels) in 3D
   • Agreed
3. Object positions are specified in pixel coordinates of the mask image.
   • [ ] We have to think about this: I also have projects where we specify in micrometer coordinates. Would it also be an option for you to make this calibrated units? Worst case we support both with a dropdown.
     • I think making it calibrated would have to advantage of forcing people to take care about and take serious their image calibration.
     • However, maybe you have a point, because the calibrated position could be computed on the fly from the image calibration.
4. I think intensity image should not be optional, rather obligatory.
   • I can agree. Related question: how many intensity images? Should we support selecting multiple columns for intensity image paths?
5. Multiple label mask images
   • Can we just treat the additional label mask images that are not used for annotation and do not correspond to the rows in the table as simple additional intensity images?

---

The reason I am asking about the global coordinate system is the following:

If a table points to multiple label mask images (which typically is the case) we need to decide how to switch the view from one label mask image to the next one, e.g., keeping B&C settings the same. I would find it easier and more informative to show all images at once (next to each other) such that moving to the next image would typically just be a lateral translation of the viewer (and zooming in on the next object if one is in annotation mode). An advantage would be that one could zoom out and get an informative overview of how many objects are already annotated and just how the whole data set looks in general.

Movies are not an issue, because my solution would have a time slider.

[tischi]

@ssgpers Regarding the name: could you live with SegmentationExplorer ?

[ssgpers]

@tischi Sure, I can live with this. Also with anything else provided I know what it is about. Just Annotation part is missing from the title now, wonder how much one would expect it within term "Explorer"

[tischi]

Just Annotation part is missing from the title now, wonder how much one would expect it within term "Explorer"

True, SegmentationAnnotator would be an alternative. Maybe better because it kind of implies that if you can annotate you also can explore?

[ssgpers]

Yes, I think this is the best potion