search

ome / openmicroscopy

OME (Open Microscopy Environment) develops open-source software and data format standards for the storage and manipulation of biological light microscopy data. A joint project between universities, research establishments and industry in Europe and the USA, OME has over 20 active researchers with strong links to the microscopy community. Funded by private and public research grants, OME has been a major force on the international microscopy stage since 2000.
https://www.openmicroscopy.org/omero
GNU General Public License v2.0
200 stars 102 forks source link

OMERO.server: JDK 17 support #6383

Open sbesson opened 7 months ago

sbesson commented 7 months ago

This issue summarizes the current known issues associated with running OMERO.server under a JDK 17 environment and discusses a set of workarounds and proposed changes.

State of Java support

As of OMERO.server 5.6.10, the requirement table of the OMERO documentation ^1 defines the current state of Java support. JDK 11 LTS released in September 2018 is the recommendation, JDK 17 LTS released in September 2021 is marked as unsupported and JDK 21 LTS which has been released in September 2023 is not yet captured.

In this table, upstream support for JDK 11 LTS is listed as ending in October 2024 i.e. 6 months from now. The reality is slightly more complex as this timeline largely depends on the OpenJDK build e.g. Oracle JDK 11 is already in extended support ^2, the RedHat build of OpenJDK will enter Extended Life Cycle Support in October ^3 while Azul Zulu will maintain builds of OpenJDK until 2032 ^4.

Introducing proper support for Java 17 (and possibly 21) in 2024 would give the community the option either to stay on JDK 11 or upgrade to a more recent JDK LTS version depending on their environment and requirements.

Encapsulation errors and workaround

The bulk of the issues associated with running OMERO.server on JDK 17 are related to the strong encapsulation introduced in Java 16 initially defined in JEP403 ^5. Various resources are available which describe these changes and their impact on downstream code e.g. ^6.

Trying to start OMERO.server-5.6.10-ice36 in a JDK 21 environment on Ubuntu 22.04 results in the following exception

2024-04-16 09:22:33,127 INFO  [                ome.services.blitz.Entry] (      main) Creating OMERO.blitz. Please wait...
2024-04-16 09:22:34,440 WARN  [                 ome.system.OmeroContext] (      main) Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'internal-ome.api.RawFileStoreSubstituter' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.RawFileStore.xml]: Cannot resolve reference to bean 'readOnlyStatus' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'readOnlyStatus' defined in class path resource [ome/services/startup.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [ome.services.util.ReadOnlyStatus]: Constructor threw exception; nested exception is java.lang.ExceptionInInitializerError
2024-04-16 09:22:34,445 ERROR [                ome.services.blitz.Entry] (      main) Error on startup.
org.springframework.beans.factory.access.BootstrapException: Unable to return specified BeanFactory instance: factory key [OMERO.blitz], from group with resource name [classpath*:beanRefContext.xml]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'OMERO.blitz' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-blitz.jar!/beanRefContext.xml]: Cannot resolve reference to bean 'ome.server' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ome.server' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/beanRefContext.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [ome.system.OmeroContext]: Constructor threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'internal-ome.api.RawFileStoreSubstituter' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.RawFileStore.xml]: Cannot resolve reference to bean 'readOnlyStatus' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'readOnlyStatus' defined in class path resource [ome/services/startup.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [ome.services.util.ReadOnlyStatus]: Constructor threw exception; nested exception is java.lang.ExceptionInInitializerError
    at org.springframework.beans.factory.access.SingletonBeanFactoryLocator.useBeanFactory(SingletonBeanFactoryLocator.java:404)
    at ome.system.OmeroContext.getInstance(OmeroContext.java:202)
    at ome.services.blitz.Entry.start(Entry.java:189)
    at ome.services.blitz.Entry.main(Entry.java:146)
...
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @fdc8126
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
    at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:199)
    at java.base/java.lang.reflect.Method.setAccessible(Method.java:193)
    at javassist.util.proxy.SecurityActions.setAccessible(SecurityActions.java:159)
    at javassist.util.proxy.DefineClassHelper$JavaOther.defineClass(DefineClassHelper.java:213)
    at javassist.util.proxy.DefineClassHelper$Java11.defineClass(DefineClassHelper.java:52)
    at javassist.util.proxy.DefineClassHelper.toClass(DefineClassHelper.java:260)
    at javassist.ClassPool.toClass(ClassPool.java:1232)
    at javassist.CtClass.toClass(CtClass.java:1384)
    at bitronix.tm.resource.jdbc.proxy.JdbcJavassistProxyFactory.generateProxyClass(JdbcJavassistProxyFactory.java:265)
    at bitronix.tm.resource.jdbc.proxy.JdbcJavassistProxyFactory.createProxyConnectionClass(JdbcJavassistProxyFactory.java:156)
    ... 110 common frames omitted

After setting omero.jvmcfg.append to --add-opens java.base/java.lang=ALL-UNNAMED and restarting the server

2024-04-16 09:25:03,635 ERROR [                ome.services.blitz.Entry] (      main) Error on startup.
org.springframework.beans.factory.access.BootstrapException: Unable to return specified BeanFactory instance: factory key [OMERO.blitz], from group with resource name [classpath*:beanRefContext.xml]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'OMERO.blitz' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-blitz.jar!/beanRefContext.xml]: Cannot resolve reference to bean 'ome.server' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ome.server' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/beanRefContext.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [ome.system.OmeroContext]: Constructor threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ome.util.MapPut#0' defined in class path resource [ome/services/hibernate.xml]: Cannot resolve reference to bean 'fullTextIndexer' while setting bean property 'object'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'fullTextIndexer' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.Search.xml]: Cannot resolve reference to bean 'eventLogLoader' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'eventLogQueue' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.Search.xml]: Cannot resolve reference to bean 'internal-ome.api.ITypes' while setting bean property 'types'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'internal-ome.api.ITypes' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.ITypes.xml]: Cannot resolve reference to bean 'internal-ome.api.IUpdate' while setting bean property 'updateService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'internal-ome.api.IUpdate' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.IUpdate.xml]: Cannot resolve reference to bean 'internal-ome.api.LocalAdmin' while setting bean property 'adminService'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'internal-ome.api.IAdmin' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.IAdmin.xml]: Cannot resolve reference to bean 'passwordProvider' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'chainedPasswordProvider' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.IAdmin.xml]: Cannot resolve reference to bean 'ldapPasswordProvider' while setting constructor argument with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ldapPasswordProvider' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.IAdmin.xml]: Cannot resolve reference to bean 'internal-ome.api.ILdap' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'internal-ome.api.ILdap' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.ILdap.xml]: Cannot resolve reference to bean 'contextSource' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contextSource' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.ILdap.xml]: Cannot resolve reference to bean 'contextSourceSwapper' while setting bean property 'targetSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contextSourceSwapper' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.ILdap.xml]: Cannot resolve reference to bean 'defaultContextSource' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'defaultContextSource' defined in URL [jar:file:/opt/omero/OMERO.server-5.6.10-ice36/lib/server/omero-server.jar!/ome/services/service-ome.api.ILdap.xml]: Bean instantiation via constructor failed; nested exception is java.lang.IllegalAccessError: class org.springframework.ldap.core.support.AbstractContextSource (in unnamed module @0x161b062a) cannot access class com.sun.jndi.ldap.LdapCtxFactory (in module java.naming) because module java.naming does not export com.sun.jndi.ldap to unnamed module @0x161b062a
    at org.springframework.beans.factory.access.SingletonBeanFactoryLocator.useBeanFactory(SingletonBeanFactoryLocator.java:404)
    at ome.system.OmeroContext.getInstance(OmeroContext.java:202)
    at ome.services.blitz.Entry.start(Entry.java:189)
    at ome.services.blitz.Entry.main(Entry.java:146)
...
Caused by: java.lang.IllegalAccessError: class org.springframework.ldap.core.support.AbstractContextSource (in unnamed module @0x161b062a) cannot access class com.sun.jndi.ldap.LdapCtxFactory (in module java.naming) because module java.naming does not export com.sun.jndi.ldap to unnamed module @0x161b062a
    at org.springframework.ldap.core.support.AbstractContextSource.<clinit>(AbstractContextSource.java:77)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
    at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:142)
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:122)
    at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:271)
    ... 171 common frames omitted

Setting omero.jvmcfg.append to --add-opens java.base/java.lang=ALL-UNNAMED --add-exports java.naming/com.sun.jndi.ldap=ALL-UNNAMED is sufficient to start OMERO.server and perform some minimal workflows: login, import, view, delete...

Minimal testing also indicated the changes above are sufficient to start OMERO.server in a JDK 21 LTS environment.

Dependency upgrades

While the paragraph above describes configuration workarounds that can be used immediately to deploy OMERO.server in a JDK 17 environment, a more robust approach would be to update the underlying libraries to versions that are not relying on features now prevented by the upstream Java changes.

Bitronix Transaction Manager

Bitronix Transaction Manager (btm) is a direct dependency of the omero-server component ^7. This library is an implementation of the Java Transaction API 1.1 (JTA). Up to OMERO.server 5.4.x, the dependency was using org.codehaus.btm:btm. In OMERO 5.5.0 and during the transition to decoupled components & the Gradle build system, this dependency was changed to com.github.marcus-nl.btm:btm:3.0.0-mk1.

The latter artifact comes from a fork of https://github.com/scalar-labs/btm which was inactive at the time but it looks like recent activity has started again on this repository. The state of these different existing implementations should be reviewed to assess whether we could upgrade this dependency to a recent version compatible with the JDK encapsulation changes.

Spring LDAP core

spring-ldap-core is a transitive dependency of the omero-server component through org.springframework.security:spring-security-ldap ^8. The usage of internal com.sun.jndi.ldap classes was fixed upstream in 2.3.4.RELEASE ^9. Upgrading manually spring-ldap-core to this version and reverting the --add-exports java.naming/com.sun.jndi.ldap=ALL-UNNAMED from the configuration above was sufficient to start the server.

At minimum, it should be possible to upgrade org.springframework.security:spring-ldap-core but this is probably a good opportunity to upgrade spring-security-ldap and review the state of the entire set of Spring dependencies as we have a mixture of versions.

/cc @jburel @pwalczysko @khaledk2 @dominikl @chris-allan @kkoz @joshmoore

jburel commented 6 months ago

As discussed on 30/04