search

IDR / idr0125-way-cellpainting

0 stars 1 forks source link

Missing rows in annotation.csv #2

Closed will-moore closed 12 months ago

will-moore commented 1 year ago

Hi @shntnu,

I just tried validating the annotation.csv file at https://github.com/IDR/idr0125-way-cellpainting/blob/main/screenA/idr0125-screenA-annotation.csv

and this reports that there is 1 row missing for Plate SQ00015207__2016-04-24T05_39_26-Measurement1, Well P1.

Any ideas what's going on here or if you have the missing metadata? Thanks,

ErinWeisbart commented 1 year ago

That is expected. We do not have data for that well. (see this comment for details)

will-moore commented 1 year ago

Thanks - as noted https://github.com/broadinstitute/lincs-cell-painting/issues/54#issuecomment-1349885429, the P1 Well is missing as intended at https://ome.github.io/ome-ngff-validator/?source=https%3A%2F%2Fcellpainting-gallery.s3.amazonaws.com%2Fcpg0004-lincs%2Fbroad%2Fimages%2F2016_04_01_a549_48hr_batch1%2Fimages_zarr%2FSQ00015207__2016-04-24T05_39_26-Measurement1.ome.zarr

However, in that Plate imported into idr0125-pilot testing server on 2022-12-26 05:19:07 isn't missing a Well at P1 and even has listed OriginalFiles in webclient

demo_2/Blitz-0-Ice.ThreadPool.Server-79/2022-12/26/03-55-40.834/SQ00015207__2016-04-24T05_39_26-Measurement1.ome.zarr/P/1/0/.zattrs
demo_2/Blitz-0-Ice.ThreadPool.Server-79/2022-12/26/03-55-40.834/SQ00015207__2016-04-24T05_39_26-Measurement1.ome.zarr/P/1/0/.zgroup

Screenshot 2023-09-18 at 14 08 31

although these are not found in the ManagedRepository itself... - P/1 not found but P/2 files are found...

(base) [wmoore@pilot-idr0125-omeroreadwrite ~]$ find /data/OMERO/ManagedRepository/demo_2/Blitz-0-Ice.ThreadPool.Server-79/2022-12/26/03-55-40.834/SQ00015207__2016-04-24T05_39_26-Measurement1.ome.zarr/P/1 -name ".zattrs" | grep "P/1"
find: ‘/data/OMERO/ManagedRepository/demo_2/Blitz-0-Ice.ThreadPool.Server-79/2022-12/26/03-55-40.834/SQ00015207__2016-04-24T05_39_26-Measurement1.ome.zarr/P/1’: No such file or directory
(base) [wmoore@pilot-idr0125-omeroreadwrite ~]$ find /data/OMERO/ManagedRepository/demo_2/Blitz-0-Ice.ThreadPool.Server-79/2022-12/26/03-55-40.834/SQ00015207__2016-04-24T05_39_26-Measurement1.ome.zarr/P/2 -name ".zattrs"
/data/OMERO/ManagedRepository/demo_2/Blitz-0-Ice.ThreadPool.Server-79/2022-12/26/03-55-40.834/SQ00015207__2016-04-24T05_39_26-Measurement1.ome.zarr/P/2/.zattrs
/data/OMERO/ManagedRepository/demo_2/Blitz-0-Ice.ThreadPool.Server-79/2022-12/26/03-55-40.834/SQ00015207__2016-04-24T05_39_26-Measurement1.ome.zarr/P/2/0/.zattrs
/data/OMERO/ManagedRepository/demo_2/Blitz-0-Ice.ThreadPool.Server-79/2022-12/26/03-55-40.834/SQ00015207__2016-04-24T05_39_26-Measurement1.ome.zarr/P/2/1/.zattrs
will-moore commented 1 year ago

The Plate imported above was downloaded with all the others on 14th December 2022 -see https://idr-redmine.openmicroscopy.org/issues/228#note-104 "Re-downloaded all 123 plates to idr0125-way-cellpainting/s3-20221212. Just 1 plate had 1 well missing"

So I wouldn't expect this missing Well to show appear in IDR when imported.

Since we aim to import OME/METADATA.ome.xml for each Plate, then use mkngff https://github.com/IDR/idr0125-way-cellpainting/issues/4 to populate the plate, lets try that with this Plate...

will-moore commented 12 months ago

When this plate is imported as METADATA.ome.xml at https://github.com/IDR/idr0125-way-cellpainting/issues/4#issuecomment-1753871872 Well P1 is missing as expected.

And that Well is also missing at https://ome.github.io/ome-ngff-validator/?source=https://cellpainting-gallery.s3.amazonaws.com/cpg0004-lincs/broad/images/2016_04_01_a549_48hr_batch1/images_zarr/SQ00015207__2016-04-24T05_39_26-Measurement1.ome.zarr

shntnu commented 11 months ago

Sorry I never got to this but sounds like you are all set 🎉