Import labels

[will-moore]

As tested with #1, we now import a single NGFF image and don't create additional images for labels.

So we need a manual way of importing labels into images (and possibly OMERO masks - as for idr0079).

copy labels/cells into cells.zarr and import...

cp -R /uod/idr/filesets/idr0138-lohoff-seqfish/20210917-Globus/ngff/TimEmbryos-102219/HybCycle_0/MMStack_Pos0.ome.zarr/labels/cells ./
mv cells MMStack_Pos0_cells.ome.zarr

$ omero import -d 18251 --transfer=ln_s --skip=all --depth=100 MMStack_Pos0_cells.ome.zarr --file /tmp/idr0138_20231107_cells.log --errs /tmp/idr0138_20231107_cells.err 

2023-11-07 13:58:39,030 320        [      main] INFO          ome.formats.importer.ImportConfig - OMERO.blitz Version: 5.7.1
2023-11-07 13:58:39,050 340        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 7.0.0 revision: 3f8b3326cb578d59bd948fb84c838ff77e9f1b08 date: 1 August 2023
2023-11-07 13:58:39,116 406        [      main] INFO   formats.importer.cli.CommandLineImporter - Setting checksum algorithm to File-Size-64
2023-11-07 13:58:39,117 407        [      main] INFO   formats.importer.cli.CommandLineImporter - Skipping thumbnails creation
2023-11-07 13:58:39,117 407        [      main] INFO   formats.importer.cli.CommandLineImporter - Skipping minimum/maximum computation
2023-11-07 13:58:39,117 407        [      main] INFO   formats.importer.cli.CommandLineImporter - Disabling upgrade check
2023-11-07 13:58:39,118 408        [      main] INFO   formats.importer.cli.CommandLineImporter - Setting transfer to ln_s
2023-11-07 13:58:39,120 410        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2023-11-07 13:58:39,495 785        [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 100 Metadata Level: MINIMUM
2023-11-07 13:58:39,521 811        [      main] ERROR                loci.formats.FormatHandler - ZarrReader loaded options file from: /opt/omero/server/idr0138/labels/MMStack_Pos0_cells.ome.zarr
2023-11-07 13:58:39,831 1121       [      main] ERROR                loci.formats.FormatHandler - ZarrReader loaded options file from: /opt/omero/server/idr0138/labels/MMStack_Pos0_cells.ome.zarr
2023-11-07 13:58:39,856 1146       [      main] ERROR                loci.formats.FormatHandler - ZarrReader getUsed files, skipPixels: false
2023-11-07 13:58:39,857 1147       [      main] ERROR                loci.formats.FormatHandler - ZarrReader fetching list of used files: /opt/omero/server/idr0138/labels/MMStack_Pos0_cells.ome.zarr
2023-11-07 13:58:39,863 1153       [      main] ERROR                loci.formats.FormatHandler - Adding to the used files list: /opt/omero/server/idr0138/labels/MMStack_Pos0_cells.ome.zarr/.zattrs
2023-11-07 13:58:39,863 1153       [      main] ERROR                loci.formats.FormatHandler - Adding to the used files list: /opt/omero/server/idr0138/labels/MMStack_Pos0_cells.ome.zarr/.zgroup
2023-11-07 13:58:39,863 1153       [      main] ERROR                loci.formats.FormatHandler - Adding to the used files list: /opt/omero/server/idr0138/labels/MMStack_Pos0_cells.ome.zarr/0/.zarray
...
2023-11-07 13:58:40,045 1335       [      main] ERROR                loci.formats.FormatHandler - Adding to the used files list: /opt/omero/server/idr0138/labels/MMStack_Pos0_cells.ome.zarr/3/.zarray
2023-11-07 13:58:40,045 1335       [      main] ERROR                loci.formats.FormatHandler - Adding to the used files list: /opt/omero/server/idr0138/labels/MMStack_Pos0_cells.ome.zarr/3/0/0/0
2023-11-07 13:58:40,045 1335       [      main] ERROR                loci.formats.FormatHandler - Adding to the used files list: /opt/omero/server/idr0138/labels/MMStack_Pos0_cells.ome.zarr/4/.zarray
2023-11-07 13:58:40,045 1335       [      main] ERROR                loci.formats.FormatHandler - Adding to the used files list: /opt/omero/server/idr0138/labels/MMStack_Pos0_cells.ome.zarr/4/0/0/0
2023-11-07 13:58:40,046 1336       [      main] INFO      ome.formats.importer.ImportCandidates - 21 file(s) parsed into 0 group(s) with 14 call(s) to setId in 540ms. (551ms total) [0 unknowns]
2023-11-07 13:58:40,096 1386       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Attempting initial SSL connection to localhost:4064
2023-11-07 13:58:40,675 1965       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Insecure connection requested, falling back
2023-11-07 13:58:41,072 2362       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Pinging session every 300s.
2023-11-07 13:58:41,086 2376       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Server: 5.6.9
2023-11-07 13:58:41,086 2376       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Client: 5.7.1
2023-11-07 13:58:41,087 2377       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Java Version: 11.0.20
2023-11-07 13:58:41,087 2377       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Name: Linux
2023-11-07 13:58:41,087 2377       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Arch: amd64
2023-11-07 13:58:41,087 2377       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Version: 3.10.0-1160.99.1.el7.x86_64
No imports found

[will-moore]

Started full import on idr-testing...

EDIT: stopped import today 9:43. 157 images imported in 17.5 hours. 6.7 mins each.

At this rate, 2912 images will take 13.5 days.

[will-moore]

With the rois.py script at https://github.com/IDR/idr0138-lohoff-seqfish/commit/7bcffabc4a3655f24d6f7e96696d3312c10c12e9

Script uses labels to add masks on the image (scaling x 4) and also creates new label images in a _seg Dataset...

Started running this at ~ 2pm today. Current rate (23 images covered) is about 2 mins per image...