search

ome / design

OME Design proposals
http://ome.github.io/design/
1 stars 15 forks source link

Import Plate/Image name metadata #57

Open chris-allan opened 7 years ago

chris-allan commented 7 years ago

The issue

The current state of how Image and Plate name are treated in the importer and the historical lack of trust put in the metadata coming out of Bio-Formats is becoming a problem.

With respect to Plate name, unless it is explicitly chosen by the user using OMERO.cli, it will be set to the Bio-Formats target filename and any Plate name metadata is ignored. This is an issue when the desired name is deep within the metadata, requiring users to perform their own metadata extraction in order to set the name. It is also particularly problematic when the Plate count being imported is large effectively necessitating that a user write shell scripts in order to import their data correctly.

The same holds true for a singular Image. If the Image is part of a multi-image fileset (MIF) its series number is used as part of the name and any Bio-Formats name metadata is ignored. This is an issue when, using images of the digital pathology domain as an example, each Image has a self describing name which ends up effectively being lost.

History and state of play

The processor that enforces the aforementioned rule on Image:

For OMERO 5.0.0 (openmicroscopy/openmicroscopy#1710) the filename, rather than the absolute path became used as the default name unless the user specifies one explicitly.

This is also when the Bio-Formats Image name metadata began being ignored for multi-image filesets. The rationale for this is not completely clear to me. Length restrictions were also put in place, substituting ellipses when the name is longer than 255 characters.

From OMERO 4.3.0, (openmicroscopy/openmicroscopy@98691605), the user specified name has defaulted to the absolute path and became just the filename in openmicroscopy/openmicroscopy#1710. The rationale, albeit not a particularly thorough, is outlined in this corresponding ticket:

The processor that enforces the aforementioned rule on Plate:

Also for OMERO 5.0.0 (openmicroscopy/openmicroscopy#577; openmicroscopy/openmicroscopy@92ec095b), the user specified Image and Plate name were unified. This had the potential unintended side effect, as the user specified Image name was the absolute path and became just the filename in openmicroscopy/openmicroscopy#1710, of always setting a user specified Plate name regardless of whether the user had conciously made that decision or not.

From OMERO 4.2.1 (openmicroscopy/openmicroscopy@87db20ab), a user specified Plate name has been possible.

Current work on the issue

No change.

OMERO 5.3.x

/cc @jburel, @joshmoore, @sbesson, @melissa-linkert, @mtbc, @will-moore

/cc @emilroz

Edits

Mon 19 Sep 2016 16:23:36 BST: Add ongoing work from @melissa-linkert, @joshmoore, @mtbc Wed 21 Sep 2016 09:22:21 BST: Reference to ongoing work on relaxing length restrictions on names

jburel commented 7 years ago

The length restriction was previously discussed but not removed this will require some db adjustments see https://trac.openmicroscopy.org/ome/ticket/11894

mtbc commented 7 years ago

http://trac.openmicroscopy.org/ome/ticket/11894 discusses the length restrictions. Happy to switch to TEXT in the DB any properties that we like. (See distinction between string and text on http://www.openmicroscopy.org/site/support/omero5.3-staging/developers/Model/EveryObject.html) Perhaps one may reasonably prioritize properties for which the corresponding OME-XML attribute is already of unrestricted length.

chris-allan commented 7 years ago

Current "named" objects, which consequently have a character varying database column of length 255 reserved to store a name property, are as follows:

Exceptions to this include ExperimenterGroup which is not "named" but has a name character varying database column of length 255.

sbesson commented 6 years ago

Update on this design issue:

mtbc commented 6 years ago

Created [trello] ignores plate names in naming fields for the issue in PixelsProcessor relating to the above community thread.

sbesson commented 5 years ago

See also https://www.openmicroscopy.org/community/viewtopic.php?f=4&p=19976#p19976 for another community use case where allowing to read the ImageName from Bio-Formats in the non-HCS behavior would be useful.

imagesc-bot commented 2 years ago

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/upload-issue-with-metadata-ome-xml-companion-file/59290/8