Bump zarr reader 0.5.1

[will-moore]

NB: this is on top of @dominikl's PR: https://github.com/IDR/deployment/pull/420/ following the discussion there, particularly https://github.com/IDR/deployment/pull/420#issuecomment-2178112947

@sbesson: I have added the list of jars under omero_client_jars but as I understand it, this will attempt to load those jars from the OME artifactory, not from maven central?

I added a variable maven_central_baseurl: "https://repo1.maven.org/maven2" and I should probably list the jars under a different variable like omero_client_maven_jars but I don't know where to update the logic that would consume that list.

cc @dominikl @jburel

[sbesson]

@will-moore I would suggest to keep exactly the logic in the current playbook but change the name of the dictionary e.g. to use zarrreader_jars

    - name: Override lib/server JARs
      become: yes
      get_url:
        url: "{{ artifactory_baseurl }}/{{ item.group }}/{{ item.name }}/{{ item.version }}/{{ item.name }}-{{ item.version }}.jar"
        dest: "{{ omero_common_basedir }}/server/OMERO.server/lib/server/{{ item.name }}.jar"
        force: yes
      with_items: "{{ zarrreader_jars }}"
      notify: restart omero-server

    - name: Override lib/client JARs
      become: yes
      get_url:
        url: "{{ artifactory_baseurl }}/{{ item.group }}/{{ item.name }}/{{ item.version }}/{{ item.name }}-{{ item.version }}.jar"
        dest: "{{ omero_common_basedir }}/server/OMERO.server/lib/client/{{ item.name }}.jar"
        force: yes
      with_items: "{{ zarrreader_jars }}"
      notify: restart omero-server

and in ansible/group_vars/omero_hosts.yml, rename omero_client_jars as zarrreader_jars and add OMEZarrReader 0.5.1 to the list

[will-moore]

@sbesson I don't see how we can have OMEZarrReader in the same list as the other dependencies since they are being loaded from different artifactories - (and the artifactory isn't specified in the list).

E.g. https://artifacts.openmicroscopy.org/artifactory/ome.releases/ome/OMEZarrReader/0.5.1/OMEZarrReader-0.5.1.jar vv https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-s3/1.12.659/aws-java-sdk-s3-1.12.659.jar

[sbesson]

If you use the virtual maven repository rather than the ome.releases repository, you should be able to retrieve both artifacts

https://artifacts.openmicroscopy.org/artifactory/maven/ome/OMEZarrReader/0.5.1/ https://artifacts.openmicroscopy.org/artifactory/maven/com/amazonaws/aws-java-sdk-s3/1.12.659/

[will-moore]

@sbesson OK thanks. Changes pushed.

[will-moore]

What's the plan for testing this? Deploy on a pilot or onto idr-next?

[will-moore]

@sbesson Does this look good now? Any way we can test it?

[sbesson]

Having not heard from the rest of the IDR team on this, I think the available options deployment wise are :

• use test122 i.e. current idr-testing.openmicroscopy.org
• use test122b i.e. current idr-next.openmicroscopy.org
• spin up prod122 and assign it to idr-next.openmicroscopy.org

I would suggest we make use the upcoming Monday meeting to make the decision

[will-moore]

Seb has deployed this on idr-next for testing... Installed omero-mkngff and ran sql scripts for idr0004 as described at https://github.com/IDR/idr-metadata/issues/696#issuecomment-2127255715

Viewing in webclient to trigger memo generation, then viewing an image from first plate: Blitz logs show blosc is missing:

2024-06-24 16:46:00,102 ERROR [        ome.services.util.ServiceHandler] (l.Server-4) java.lang.Error:  Wrapped Exception: (java.lang.NoClassDefFoundError):
Could not initialize class org.blosc.IBloscDll
java.lang.NoClassDefFoundError: Could not initialize class org.blosc.IBloscDll
    at com.bc.zarr.CompressorFactory$BloscCompressor.cbufferSizes(CompressorFactory.java:352)
    at com.bc.zarr.CompressorFactory$BloscCompressor.uncompress(CompressorFactory.java:338)
    at com.bc.zarr.chunk.ChunkReaderWriterImpl_Short.read(ChunkReaderWriterImpl_Short.java:55)
    at com.bc.zarr.ZarrArray.read(ZarrArray.java:278)
    at com.bc.zarr.ZarrArray.read(ZarrArray.java:255)
    at loci.formats.services.JZarrServiceImpl.readBytes(JZarrServiceImpl.java:241)

[sbesson]

To be specific, I deployed the combination of https://github.com/IDR/deployment/pull/427, https://github.com/IDR/deployment/pull/424 and https://github.com/IDR/deployment/pull/423 against test122b.

The error above suggests we are missing the blosc dependency which brings us back to https://github.com/IDR/deployment/pull/420#issuecomment-2051362554. Could you update this PR to bump the role version and I'll redeploy tomorrow morning.

[will-moore]

@sbesson Sorry, I'm not sure where/how to "update this PR to bump the role version".

[will-moore]

We still have this in deployment/ansible/requirements.yml

- src: ome.omero_server
  version: 6.0.0

Is that what you meant? In the discussion on that at https://github.com/IDR/deployment/pull/420#pullrequestreview-2052198000 you say "the plan for prod122 is to bump OMERO.server to the just released 5.6.11" Which seems to mean installing an older version rather than bumping the version?

[sbesson]

These are two independent versions. https://github.com/IDR/deployment/pull/420#issuecomment-2051362554 is referring to the version of the Ansible role as the blosc installation was introduced in https://github.com/ome/ansible-role-omero-server/pull/75. https://github.com/IDR/deployment/pull/420#pullrequestreview-2052198000 is referring to the value of idr_omero_server_release which specifies the version of the OMERO.server binary deployed by the role and is already updated by this PR.

[will-moore]

@sbesson understood now, thanks. Bumped the role version in 8caf45e

[will-moore]

@sbesson I don't know if this is expected to be fixed now, but I'm still seeing Blosc errors on test122b-omeroreadwrite when trying to view NGFF data (idr0004)?

[sbesson]

Definitely not expected but one thing that comes to mind is that the playbook didn't restart the omero-server service (since nothing changed). Can you try running sudo systemctl restart omero-server and see if the blosc issue persists?

[will-moore]

Great - yes that fixed it, Thanks. I didn't need to do that on e.g. omeroreadonly-3 after I restarted omeroreadwrite server. Is that effectively the same as sudo service omero-server restart?

[sbesson]

Great - yes that fixed it, Thanks. I didn't need to do that on e.g. omeroreadonly-3 after I restarted omeroreadwrite server.

These might have been restarted as per the changed=6 above. Only the omeroreadwrite server has not. I don't think it's worth investigating why this is different at this stage.

Is that effectively the same as sudo service omero-server restart?

Yes exactly.