We have deployed wso2 apim 3.1.0 image on GKE (1.21.10-gke.2000) managed service and currently we are seeing wired behavior during wso2 apim container startup - IdentityUtil Class Definition Not Found Error. The required jar and IdentityUtil classes are present in class path.
To our surprise, the same wso2 apim 3.1.0 image works in older version of GKE version(1.21.9-gke.1002) and currently we have no clue on what is causing in latest gke(1.21.10-gke.2000) version.
[2022-05-09 06:15:42,075] ERROR - [/accountrecoveryendpoint] Exception sending context initialized event to listener instance of class [org.wso2.carbon.identity.mgt.endpoint.util.listener.IdentityManagementEndpointContextListener]
java.lang.NoClassDefFoundError: org/wso2/carbon/identity/core/util/IdentityUtil
at org.wso2.carbon.identity.mgt.endpoint.util.IdentityManagementServiceUtil.init(IdentityManagementServiceUtil.java:127) ~[org.wso2.carbon.identity.mgt.endpoint.util_5.17.5.jar:?]
at org.wso2.carbon.identity.mgt.endpoint.util.listener.IdentityManagementEndpointContextListener.contextInitialized(IdentityManagementEndpointContextListener.java:34) ~[org.wso2.carbon.identity.mgt.endpoint.util_5.17.5.jar:?]
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4685) [tomcat_9.0.31.wso2v1.jar:?]
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5146) [tomcat_9.0.31.wso2v1.jar:?]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) [tomcat_9.0.31.wso2v1.jar:?]
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:717) [tomcat_9.0.31.wso2v1.jar:?]
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:690) [tomcat_9.0.31.wso2v1.jar:?]
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705) [tomcat_9.0.31.wso2v1.jar:?]
at org.wso2.carbon.tomcat.internal.CarbonTomcat.addWebApp(CarbonTomcat.java:303) [org.wso2.carbon.tomcat_4.6.0.jar:?]
at org.wso2.carbon.tomcat.internal.CarbonTomcat.addWebApp(CarbonTomcat.java:209) [org.wso2.carbon.tomcat_4.6.0.jar:?]
at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.handleWebappDeployment(TomcatGenericWebappsDeployer.java:255) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?]
at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.handleExplodedWebappDeployment(TomcatGenericWebappsDeployer.java:243) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?]
at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.handleHotDeployment(TomcatGenericWebappsDeployer.java:173) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?]
at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.deploy(TomcatGenericWebappsDeployer.java:140) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?]
at org.wso2.carbon.webapp.mgt.AbstractWebappDeployer.deployThisWebApp(AbstractWebappDeployer.java:224) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?]
at org.wso2.carbon.webapp.mgt.AbstractWebappDeployer.deploy(AbstractWebappDeployer.java:114) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?]
at org.wso2.carbon.webapp.deployer.WebappDeployer.deploy(WebappDeployer.java:42) [org.wso2.carbon.webapp.deployer_4.10.1.jar:?]
at org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136) [axis2_1.6.1.wso2v41.jar:?]
at org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:807) [axis2_1.6.1.wso2v41.jar:?]
at org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:144) [axis2_1.6.1.wso2v41.jar:?]
at org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377) [axis2_1.6.1.wso2v41.jar:?]
at org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254) [axis2_1.6.1.wso2v41.jar:?]
at org.apache.synapse.Axis2SynapseController.deployMediatorExtensions(Axis2SynapseController.java:783) [synapse-core_2.1.7.wso2v147.jar:2.1.7-wso2v147]
at org.apache.synapse.Axis2SynapseController.createSynapseEnvironment(Axis2SynapseController.java:407) [synapse-core_2.1.7.wso2v147.jar:2.1.7-wso2v147]
at org.apache.synapse.ServerManager.start(ServerManager.java:183) [synapse-core_2.1.7.wso2v147.jar:2.1.7-wso2v147]
at org.wso2.carbon.mediation.initializer.ServiceBusInitializer.initESB(ServiceBusInitializer.java:371) [org.wso2.carbon.mediation.initializer_4.7.35.jar:?]
at org.wso2.carbon.mediation.initializer.ServiceBusInitializer.activate(ServiceBusInitializer.java:170) [org.wso2.carbon.mediation.initializer_4.7.35.jar:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?]
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?]
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?]
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?]
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?]
at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?]
at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?]
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:113) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:985) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:234) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:151) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:866) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:804) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:228) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.internal.framework.BundleContextImpl.registerService(BundleContextImpl.java:525) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.eclipse.osgi.internal.framework.BundleContextImpl.registerService(BundleContextImpl.java:544) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?]
at org.wso2.carbon.inbound.endpoint.persistence.service.InboundEndpointPersistenceServiceDSComponent.activate(InboundEndpointPersistenceServiceDSComponent.java:50) [org.wso2.carbon.inbound.endpoint.persistence_4.7.35.jar:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
Description:
We have deployed wso2 apim 3.1.0 image on GKE (1.21.10-gke.2000) managed service and currently we are seeing wired behavior during wso2 apim container startup - IdentityUtil Class Definition Not Found Error. The required jar and IdentityUtil classes are present in class path.
To our surprise, the same wso2 apim 3.1.0 image works in older version of GKE version(1.21.9-gke.1002) and currently we have no clue on what is causing in latest gke(1.21.10-gke.2000) version.
Similar issue found:
https://stackoverflow.com/questions/72158227/identityutil-class-definition-not-found-error-in-wso2-api-manager-4-1-startup
Affected Product Versions:
WSO2 APIM 3.1.0 GKE (1.21.10-gke.2000)
Working Product Versions:
WSO2 APIM 3.1.0 GKE (1.21.9-gke.1002)
Steps to reproduce:
WSO2 apim error stacktrace:
[2022-05-09 06:15:42,075] ERROR - [/accountrecoveryendpoint] Exception sending context initialized event to listener instance of class [org.wso2.carbon.identity.mgt.endpoint.util.listener.IdentityManagementEndpointContextListener] java.lang.NoClassDefFoundError: org/wso2/carbon/identity/core/util/IdentityUtil at org.wso2.carbon.identity.mgt.endpoint.util.IdentityManagementServiceUtil.init(IdentityManagementServiceUtil.java:127) ~[org.wso2.carbon.identity.mgt.endpoint.util_5.17.5.jar:?] at org.wso2.carbon.identity.mgt.endpoint.util.listener.IdentityManagementEndpointContextListener.contextInitialized(IdentityManagementEndpointContextListener.java:34) ~[org.wso2.carbon.identity.mgt.endpoint.util_5.17.5.jar:?] at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4685) [tomcat_9.0.31.wso2v1.jar:?] at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5146) [tomcat_9.0.31.wso2v1.jar:?] at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) [tomcat_9.0.31.wso2v1.jar:?] at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:717) [tomcat_9.0.31.wso2v1.jar:?] at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:690) [tomcat_9.0.31.wso2v1.jar:?] at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705) [tomcat_9.0.31.wso2v1.jar:?] at org.wso2.carbon.tomcat.internal.CarbonTomcat.addWebApp(CarbonTomcat.java:303) [org.wso2.carbon.tomcat_4.6.0.jar:?] at org.wso2.carbon.tomcat.internal.CarbonTomcat.addWebApp(CarbonTomcat.java:209) [org.wso2.carbon.tomcat_4.6.0.jar:?] at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.handleWebappDeployment(TomcatGenericWebappsDeployer.java:255) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?] at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.handleExplodedWebappDeployment(TomcatGenericWebappsDeployer.java:243) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?] at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.handleHotDeployment(TomcatGenericWebappsDeployer.java:173) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?] at org.wso2.carbon.webapp.mgt.TomcatGenericWebappsDeployer.deploy(TomcatGenericWebappsDeployer.java:140) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?] at org.wso2.carbon.webapp.mgt.AbstractWebappDeployer.deployThisWebApp(AbstractWebappDeployer.java:224) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?] at org.wso2.carbon.webapp.mgt.AbstractWebappDeployer.deploy(AbstractWebappDeployer.java:114) [org.wso2.carbon.webapp.mgt_4.10.1.jar:?] at org.wso2.carbon.webapp.deployer.WebappDeployer.deploy(WebappDeployer.java:42) [org.wso2.carbon.webapp.deployer_4.10.1.jar:?] at org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136) [axis2_1.6.1.wso2v41.jar:?] at org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:807) [axis2_1.6.1.wso2v41.jar:?] at org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:144) [axis2_1.6.1.wso2v41.jar:?] at org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377) [axis2_1.6.1.wso2v41.jar:?] at org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254) [axis2_1.6.1.wso2v41.jar:?] at org.apache.synapse.Axis2SynapseController.deployMediatorExtensions(Axis2SynapseController.java:783) [synapse-core_2.1.7.wso2v147.jar:2.1.7-wso2v147] at org.apache.synapse.Axis2SynapseController.createSynapseEnvironment(Axis2SynapseController.java:407) [synapse-core_2.1.7.wso2v147.jar:2.1.7-wso2v147] at org.apache.synapse.ServerManager.start(ServerManager.java:183) [synapse-core_2.1.7.wso2v147.jar:2.1.7-wso2v147] at org.wso2.carbon.mediation.initializer.ServiceBusInitializer.initESB(ServiceBusInitializer.java:371) [org.wso2.carbon.mediation.initializer_4.7.35.jar:?] at org.wso2.carbon.mediation.initializer.ServiceBusInitializer.activate(ServiceBusInitializer.java:170) [org.wso2.carbon.mediation.initializer_4.7.35.jar:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?] at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?] at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?] at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?] at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?] at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?] at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222) [org.eclipse.equinox.ds_1.4.400.v20160226-2036.jar:?] at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:113) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?] at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:985) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?] at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:234) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?] at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:151) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?] at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:866) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?] at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:804) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?] at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?] at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:228) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?] at org.eclipse.osgi.internal.framework.BundleContextImpl.registerService(BundleContextImpl.java:525) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?] at org.eclipse.osgi.internal.framework.BundleContextImpl.registerService(BundleContextImpl.java:544) [org.eclipse.osgi_3.14.0.v20190517-1309.jar:?] at org.wso2.carbon.inbound.endpoint.persistence.service.InboundEndpointPersistenceServiceDSComponent.activate(InboundEndpointPersistenceServiceDSComponent.java:50) [org.wso2.carbon.inbound.endpoint.persistence_4.7.35.jar:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
Related Issues:
https://stackoverflow.com/questions/72158227/identityutil-class-definition-not-found-error-in-wso2-api-manager-4-1-startup
Can somone help me to understand why wso2 apim stopped working on gke upgraded version?
Please help..