Revision of OME Data Model Diagrams

[gusferguson]

The data model documentation includes a number of diagrams generated from a scatter of Omnigraffle files making updating and refining arduous.

As part of the refresh of the website/documentation, the diagrams have been standardised and are so they can all be generated from a single Omnigraffle canvas optimising these of shared elements. A README is provided. This is with the aim of:

• making future updating easier
• enabling easy construction of a custom diagram for presentation etc.

After initial consultation with @mtbc and @hflynn the refreshed diagrams are presented for general comment and feedback.

On consideration there will probably be several areas that need updating and there may be gaps identified where new diagrams are required.

Please comment and feedback.

The diagrams presented are mostly from the current Overview page:

http://www.openmicroscopy.org/site/support/ome-model/developers/model-overview.html

There may be other diagrams that need to be added. The existing text is used. The top-level object order was revamped into a (hopefully) more logical order. Colours were standardised and fewer used. A standard width of 700 px is used to keep all objects across the diagrams the same size.

---

Current Data Model overview

The diagrams below illustrate some aspects of the model and further details are given on the following pages.

Top Level Objects

Image branch of the OME Schema

Instrument branch of the OME Schema

Location of Filters and Light Paths in the OME Model

Organizational structures (Project, Dataset, Group, Experiment, Experimenter) of the OME Model

Ownership relations for Project, Dataset and Experimenter

Region Of Interest branch of the OME Model

Shapes in the OME Model

HCS structures (Screen, Plate and Well) of the OME Model

StructuredAnnotation branch of the OME Model

All points in the OME Model that can be Annotated

Image Basic

Instrument - Simpler

[joshmoore]

I'm liking these a lot, @gusferguson! The color palette is soothing. One RFE'ish comment: though the e.g. the HCS diagram is accurate, I think it still fails to convey to people the multi-dimensional nature of what's going on there. (Similarly, I recently had someone look at a Plate layout and not realize that each each what actually a movie)

It might be that higher-dimensional relationships need something more cartoon-y to get the structure across. However, that will introduce another level of images to support. If there's any simple way we could convey some of that structure via these official model diagrams, I'd likely vote for it.

[sbesson]

Thanks @gusferguson. Focusing especially at the Region of Interest diagrams post 2016-06 work, I wonder whether the list of shapes should not be migrated to the Shape diagram in between the Shape and the attributes. In addition to a symmetry with the Structured Annotation diagram (even though the model inheritance is technically different), it would also simplify the Region of Interest diagram. On the latter, would there be a case for adding ROIRef to Folder as well?

[gusferguson]

@joshmoore @sbesson

Thanks for the feedback. All aspects are completely open at this stage - this was really to start the discussion about what is really needed and useful to have. Most of the effort has gone into making the diagrams easy to edit and adapt. It would be really helpful to have some rough sketches of what you are talking about - perhaps we can schedule a chat some time when everyone has had a chance to have a look - and either do screenshare swapping of sketches or you can send scans of rough doodles.

[mtbc]

The only model diagrams I know of are those on pages listed at "The Data Model in detail" on http://www.openmicroscopy.org/site/support/ome-model/#the-data-model-in-detail.

[mtbc]

Interested readers of this issue may also wish to review the "Data Model detail" column at https://trello.com/b/fMBThxpK/docs-comms in case any points raised there can be addressed here.

[gusferguson]

@mtbc - thanks - I have commented on most of the cards there.

As-is revision of http://www.openmicroscopy.org/site/support/ome-model/_images/FilterSet-LightPath.png (https://trello.com/c/c6qKsnlm/494-filter-attributes) although I can see why you may want to ditch it!

[mtbc]

"Emmission" typo in above. (But, yeah, maybe ditch anyway!)

[mtbc]

Interesting cardinality issues. For instance, why does "Image" to "ROIRef" get a "lots of ROIRefs" treatment but "Pixels" to "Channel" doesn't? (Probably nobody knows, unless there's a clue in the narrative context in the text around the diagram.)

[hflynn]

The times where we indicate possible multiples are not consistent at all @mtbc

[hflynn]

I've made a bunch of comments on the related cards but to summarize:

• the new simplified colour scheme is definitely an improvement
• the top level block looks better and I like having it as a consistent feature, I think it improves readability (however, in cases like the FilterSet-LightPath above where it makes the diagram much taller, we could omit most of the faded-out objects with some sort of representation that it's been 'snipped')
• I'm still not clear if we have consistency on what dashed lines mean between different diagrams
• I think we've lost the bit of the ROI diagram that was most graphical and kept the bits which are essentially lists reproduced from the text but with less helpful detail
• i like the new simplified annotation points diagram and would keep that rather than replacing with text

[hflynn]

SPW diagrams still to tackle - I like Josh's idea on https://trello.com/c/nUu7hExY/495-multiple-cross-links of completely changing the way the file structure is represented, I don't think that needs to use the same linked boxes structure as these other diagrams which are reflecting the schema structure

[hflynn]

Previous filter attributes diagram has dotted lines to indicate them being optional, as described in the text - this element has been removed from the new version you've added in comments @gusferguson

[mtbc]

the bit of the ROI diagram that was most graphical

... that's the only bit of it I've much interest in keeping, though could possibly be split out into several mini-diagrams.

[mtbc]

@joshmoore's presentation slide needs makes me wonder if it would be good to step back and have people enumerate what they want to get from this documentation that isn't already covered in the generated documentation. There's certainly some, but what? Our manually curated text has maintenance overhead and can link into the generated documentation as appropriate. As context I'll try to bring the Trello cards together with @gusferguson's work to explain what my "master plan" for these pages would be, of course carrying the weight of but a 2¢ proposal, and I am not sure what more should be in our model detail pages yet isn't.

[joshmoore]

One presentation example that might also be worth looking at is the recent MULTIMOT deck. I tried to create a number of slides using a box metaphor to show how model objects map from a proprietary format to the common model. That's the most ambitious I've been in trying to explain the model.

[mtbc]

I've laid out some thoughts on cards but I don't mind if people don't bite on many. Looking further through our current pages I suppose, in line with my ideas so far, I'd probably be controversially radical in cutting them (and their diagrams) down but if the maintenance burden isn't too bad and others are happy then it may not be worth such effort. So, for others, especially @gusferguson: what's your thinking from here: Do you think we should move on to a PR wherein new-style diagrams substitute for the current (maybe one or two minor text adjustments would need to go along with that) comprehensively enough that we don't reuse any from the old colorful style, or are there larger changes beyond these so far that you think important to making these pages more useful for developers for 5.3.0 (though perhaps not in the initial PR)?

[gusferguson]

Added version of the original shape descriptions to the diagrams above - as requested in https://trello.com/c/44DkDy1H/490-roi-diagram and @hflynn comment above.

[gusferguson]

Adding version of diagram mentioned in https://trello.com/c/k2yRlamP/496-tiled-image-diagram that ties into the model diagrams.

[mtbc]

Cool. It's okay to omit the stagelabel / stageposition stuff?

[gusferguson]

@mtbc - I left it out because it didn't map to any model diagrams I had done but happy to do version with this added in if people think it a good idea.

[gusferguson]

Representation 0f 5D data:

[joshmoore]

I'm really liking these, @gusferguson. For those following along, this last one is loosely based on https://www.researchgate.net/profile/Ian_Harper2/publication/11490539/figure/fig2/AS:277471996989442@1443165887647/Figure-2-The-5D-image-A-The-scope-of-multidimensional-data-sets-Each-single-2D.png

My primary "concern" is that it's a little unclear how to mentally merge the image volume with the well volume. i.e. is the cylinder in the second image (non-HCS) the same as that in the first image (non-HCS)?

I know it's not completely realistic, but perhaps in the well case, there could be fewer cells and each of them could be captured as an image? (Of course, if we ever want to capture the concept of ROI in this same graphic it'll get harder...)

[gusferguson]

@joshmoore - happy to try and work towards what you need, once I understand exactly what it is. Unfortunately won't have time to do it for the talk tomorrow. Hope these suffice for the moment.

[gusferguson]

@joshmoore - updated graffle file is on squig if you want to play yourself.

[hflynn]

Are all these on squig @gusferguson? We should probably get them into the docs before we migrate them to the new ome-model repo

[gusferguson]

@hflynn - they are all on squig: /ome/team/rkferguson/2 Scoping/44 Model Diagrams/ along with the Omnigraffle file they are created from. You will need to keep that somewhere as well.

[hflynn]

Thanks @gusferguson I'll get a PR open and we can discuss whether to store the graffle file in the repo or elsewhere.

[hflynn]

That's the live docs updated with the new diagrams, thanks all for your work on this!