search

ome / ZarrReader

Other
5 stars 8 forks source link

Use canonical path for calls to ZarrService #57

Closed dgault closed 1 year ago

dgault commented 1 year ago

This PR is open for the purpose of testing the OMERO symlink issues.

Although we do not want to be using canonical paths within the reader, it seems that doing so when calling jzarr, does seem to resolve the issues and exceptions seen. I would expect a better long term solution would be to resolve this either at the JZarr level or within the ZarrService which call JZarr.

dgault commented 1 year ago

Unit test failures, investigating now

snoopycrimecop commented 1 year ago

Conflicting PR. Removed from build BIOFORMATS-push#535. See the console output for more details. Possible conflicts:

--conflicts

joshmoore commented 1 year ago

Ah, @dgault, looks like we will need to start excluding or merging things.

snoopycrimecop commented 1 year ago

Conflicting PR. Removed from build BIOFORMATS-push#29. See the console output for more details. Possible conflicts:

--conflicts Conflict resolved in build BIOFORMATS-push#537. See the console output for more details.

joshmoore commented 1 year ago

The exception is gone, but for some reason the files are still not being detected. For /tmp/b.ome.zarr which is a symlink to /tmp/a.ome.zarr:

(omero) [jamoore@pilot-idr0125-omeroreadwrite ~]$ omero import -f --depth=20  /tmp/b.ome.zarr/
Using OMERO.java-5.6.3-346-5b8271d-ice36-b236
2023-07-05 09:34:00,594 285        [      main] INFO          ome.formats.importer.ImportConfig - OMERO.blitz Version: 5.6.3-SNAPSHOT
2023-07-05 09:34:00,611 302        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 6.13.1-SNAPSHOT revision: c86fa185c70978f4d9eeacf5177c4fc0c3fa6343 date: 5 July 2023
2023-07-05 09:34:00,699 390        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2023-07-05 09:34:01,041 732        [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 20 Metadata Level: MINIMUM
2023-07-05 09:34:01,716 1407       [      main] INFO      ome.formats.importer.ImportCandidates - 17 file(s) parsed into 0 group(s) with 5 call(s) to setId in 668ms. (675ms total) [0 unknowns]

as opposed to /tmp/a.ome.zarr:

(omero) [jamoore@pilot-idr0125-omeroreadwrite ~]$ omero import -f --depth=20  /tmp/a.ome.zarr/
Using OMERO.java-5.6.3-346-5b8271d-ice36-b236
2023-07-05 09:33:14,949 330        [      main] INFO          ome.formats.importer.ImportConfig - OMERO.blitz Version: 5.6.3-SNAPSHOT
2023-07-05 09:33:14,968 349        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 6.13.1-SNAPSHOT revision: c86fa185c70978f4d9eeacf5177c4fc0c3fa6343 date: 5 July 2023
2023-07-05 09:33:15,049 430        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2023-07-05 09:33:15,445 826        [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 20 Metadata Level: MINIMUM
2023-07-05 09:33:16,004 1385       [      main] INFO      ome.formats.importer.ImportCandidates - 17 file(s) parsed into 1 group(s) with 1 call(s) to setId in 555ms. (559ms total) [0 unknowns]
#======================================
# Group: /tmp/a.ome.zarr/OME/METADATA.ome.xml SPW: false Reader: loci.formats.in.ZarrReader
/tmp/a.ome.zarr/OME/METADATA.ome.xml
/tmp/a.ome.zarr/OME/.zgroup
/tmp/a.ome.zarr/OME/.zattrs
/tmp/a.ome.zarr/.zgroup
/tmp/a.ome.zarr/.zattrs
/tmp/a.ome.zarr/0/0/.zarray
/tmp/a.ome.zarr/0/0/0/0/0/0/0
/tmp/a.ome.zarr/0/1/.zarray
/tmp/a.ome.zarr/0/1/0/0/0/0/0
/tmp/a.ome.zarr/0/.zgroup
/tmp/a.ome.zarr/0/.zattrs
/tmp/a.ome.zarr/1/0/.zarray
/tmp/a.ome.zarr/1/0/0/0/0/0/0
/tmp/a.ome.zarr/1/1/.zarray
/tmp/a.ome.zarr/1/1/0/0/0/0/0
/tmp/a.ome.zarr/1/.zgroup
/tmp/a.ome.zarr/1/.zattrs
joshmoore commented 1 year ago

From a private message to David, showinf was showing this for a symlinked directory:

filename = /tmp/b.ome.zarr/.zattrs
************ invalid used files list ************
Used files = []
joshmoore commented 1 year ago

With the new commit:

filename = /tmp/b.ome.zarr/.zattrs
Used files:
    /tmp/b.ome.zarr/OME/METADATA.ome.xml
    /tmp/b.ome.zarr/OME/.zgroup
    /tmp/b.ome.zarr/OME/.zattrs
    /tmp/b.ome.zarr/.zgroup
    /tmp/b.ome.zarr/.zattrs
    /tmp/b.ome.zarr/0/0/.zarray
    /tmp/b.ome.zarr/0/0/0/0/0/0/0
    /tmp/b.ome.zarr/0/1/.zarray
    /tmp/b.ome.zarr/0/1/0/0/0/0/0
    /tmp/b.ome.zarr/0/.zgroup
    /tmp/b.ome.zarr/0/.zattrs
    /tmp/b.ome.zarr/1/0/.zarray
    /tmp/b.ome.zarr/1/0/0/0/0/0/0
    /tmp/b.ome.zarr/1/1/.zarray
    /tmp/b.ome.zarr/1/1/0/0/0/0/0
    /tmp/b.ome.zarr/1/.zgroup
    /tmp/b.ome.zarr/1/.zattrs

and

(omero) [jamoore@pilot-idr0125-omeroreadwrite jars]$ omero import -f --depth=20  /tmp/b.ome.zarr/
Using OMERO.java-5.6.3-346-5b8271d-ice36-b236
2023-07-06 09:29:18,640 299        [      main] INFO          ome.formats.importer.ImportConfig - OMERO.blitz Version: 5.6.3-SNAPSHOT
2023-07-06 09:29:18,656 315        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 6.13.1-SNAPSHOT revision: c86fa185c70978f4d9eeacf5177c4fc0c3fa6343 date: 5 July 2023
2023-07-06 09:29:18,725 384        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2023-07-06 09:29:19,074 733        [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 20 Metadata Level: MINIMUM
2023-07-06 09:29:19,620 1279       [      main] INFO      ome.formats.importer.ImportCandidates - 17 file(s) parsed into 1 group(s) with 1 call(s) to setId in 541ms. (546ms total) [0 unknowns]
#======================================
# Group: /tmp/b.ome.zarr/OME/METADATA.ome.xml SPW: false Reader: loci.formats.in.ZarrReader
/tmp/b.ome.zarr/OME/METADATA.ome.xml
/tmp/b.ome.zarr/OME/.zgroup
/tmp/b.ome.zarr/OME/.zattrs
/tmp/b.ome.zarr/.zgroup
/tmp/b.ome.zarr/.zattrs
/tmp/b.ome.zarr/0/0/.zarray
/tmp/b.ome.zarr/0/0/0/0/0/0/0
/tmp/b.ome.zarr/0/1/.zarray
/tmp/b.ome.zarr/0/1/0/0/0/0/0
/tmp/b.ome.zarr/0/.zgroup
/tmp/b.ome.zarr/0/.zattrs
/tmp/b.ome.zarr/1/0/.zarray
/tmp/b.ome.zarr/1/0/0/0/0/0/0
/tmp/b.ome.zarr/1/1/.zarray
/tmp/b.ome.zarr/1/1/0/0/0/0/0
/tmp/b.ome.zarr/1/.zgroup
/tmp/b.ome.zarr/1/.zattrs

:+1:

snoopycrimecop commented 1 year ago

Conflicting PR. Removed from build BIOFORMATS-push#31. See the console output for more details. Possible conflicts:

--conflicts

snoopycrimecop commented 1 year ago

Conflicting PR. Removed from build BIOFORMATS-push#27. See the console output for more details. Possible conflicts:

--conflicts