search

gravitee-io / issues

Gravitee.io - API Platform - Issues
64 stars 26 forks source link

Errors during installation of Gravitee Gateway in Kubernetes #8337

Open sanjimoh opened 2 years ago

sanjimoh commented 2 years ago

Hi Team, I am trying to setup Gravitee on Kubernetes following the documentation from here.

Following is my installation stack - stack

Status of the pods appears to be running but the gateway pod has errors - gw-pods

gw-pod-init-false

The error in the Gateway pod appears to be happening when the gateway pod is installing the ingress plugin. Could you please support here?

05:22:09.949 [graviteeio-node] [] INFO  i.g.n.p.s.h.ServicePluginHandler - Install plugin: ingress [io.gravitee.gateway.services.kube.KubeSyncService]
05:22:11.150 [graviteeio-node] [] WARN  o.s.c.a.AnnotationConfigApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'io.gravitee.gateway.services.kube.KubeSyncService': Unsatisfied dependency expressed through field 'servicesManager'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'servicesResourceManager': Unsatisfied dependency expressed through field 'graviteeServices'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'servicesResourceServiceImpl': Lookup method resolution failed; nested exception is java.lang.IllegalStateException: Failed to introspect Class [io.gravitee.gateway.services.kube.services.impl.ServicesResourceServiceImpl] from ClassLoader [java.net.FactoryURLClassLoader@6892d403]
05:22:11.154 [graviteeio-node] [] ERROR i.g.p.c.i.PluginContextFactoryImpl - Unable to refresh plugin context
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'io.gravitee.gateway.services.kube.KubeSyncService': Unsatisfied dependency expressed through field 'servicesManager'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'servicesResourceManager': Unsatisfied dependency expressed through field 'graviteeServices'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'servicesResourceServiceImpl': Lookup method resolution failed; nested exception is java.lang.IllegalStateException: Failed to introspect Class [io.gravitee.gateway.services.kube.services.impl.ServicesResourceServiceImpl] from ClassLoader [java.net.FactoryURLClassLoader@6892d403]
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:659)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:639)
    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:953)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
    at io.gravitee.plugin.core.internal.PluginContextFactoryImpl.create(PluginContextFactoryImpl.java:59)
    at io.gravitee.plugin.core.internal.PluginContextFactoryImpl.create(PluginContextFactoryImpl.java:73)
    at io.gravitee.plugin.core.api.AbstractSpringPluginHandler.handle(AbstractSpringPluginHandler.java:33)
    at io.gravitee.plugin.core.api.AbstractPluginHandler.handle(AbstractPluginHandler.java:65)
    at io.gravitee.plugin.core.internal.PluginEventListener.lambda$deployPlugin$8(PluginEventListener.java:125)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source)
    at java.base/java.util.stream.ReferencePipeline$2$1.accept(Unknown Source)
    at java.base/java.util.Iterator.forEachRemaining(Unknown Source)
    at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Unknown Source)
    at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
    at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(Unknown Source)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(Unknown Source)
    at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
    at java.base/java.util.stream.ReferencePipeline.forEach(Unknown Source)
    at io.gravitee.plugin.core.internal.PluginEventListener.deployPlugin(PluginEventListener.java:123)
    at io.gravitee.plugin.core.internal.PluginEventListener.lambda$deployPlugins$5(PluginEventListener.java:108)
    at java.base/java.util.ArrayList.forEach(Unknown Source)
    at io.gravitee.plugin.core.internal.PluginEventListener.deployPlugins(PluginEventListener.java:108)
    at io.gravitee.plugin.core.internal.PluginEventListener.onEvent(PluginEventListener.java:67)
    at io.gravitee.common.event.impl.EventManagerImpl.publishEvent(EventManagerImpl.java:43)
    at io.gravitee.common.event.impl.EventManagerImpl.publishEvent(EventManagerImpl.java:35)
    at io.gravitee.plugin.core.internal.PluginRegistryImpl.init(PluginRegistryImpl.java:115)
    at io.gravitee.plugin.core.internal.PluginRegistryImpl.doStart(PluginRegistryImpl.java:91)
    at io.gravitee.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:32)
    at io.gravitee.node.container.AbstractNode.startComponents(AbstractNode.java:202)
    at io.gravitee.node.container.AbstractNode.doStart(AbstractNode.java:76)
    at io.gravitee.gateway.standalone.node.GatewayNode.doStart(GatewayNode.java:78)
    at io.gravitee.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:32)
    at io.gravitee.node.container.AbstractContainer.doStart(AbstractContainer.java:106)
    at io.gravitee.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:32)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.base/java.lang.reflect.Method.invoke(Unknown Source)
    at io.gravitee.gateway.standalone.boostrap.Bootstrap.start(Bootstrap.java:154)
    at io.gravitee.gateway.standalone.boostrap.Bootstrap.main(Bootstrap.java:163)
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'servicesResourceManager': Unsatisfied dependency expressed through field 'graviteeServices'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'servicesResourceServiceImpl': Lookup method resolution failed; nested exception is java.lang.IllegalStateException: Failed to introspect Class [io.gravitee.gateway.services.kube.services.impl.ServicesResourceServiceImpl] from ClassLoader [java.net.FactoryURLClassLoader@6892d403]
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:659)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:639)
    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:119)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:399)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1431)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1389)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1309)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:656)
    ... 50 common frames omitted
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'servicesResourceServiceImpl': Lookup method resolution failed; nested exception is java.lang.IllegalStateException: Failed to introspect Class [io.gravitee.gateway.services.kube.services.impl.ServicesResourceServiceImpl] from ClassLoader [java.net.FactoryURLClassLoader@6892d403]
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.determineCandidateConstructors(AutowiredAnnotationBeanPostProcessor.java:289)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.determineConstructorsFromBeanPostProcessors(AbstractAutowireCapableBeanFactory.java:1302)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1219)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
    at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1389)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1309)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:656)
    ... 64 common frames omitted
Caused by: java.lang.IllegalStateException: Failed to introspect Class [io.gravitee.gateway.services.kube.services.impl.ServicesResourceServiceImpl] from ClassLoader [java.net.FactoryURLClassLoader@6892d403]
    at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:485)
    at org.springframework.util.ReflectionUtils.doWithLocalMethods(ReflectionUtils.java:321)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.determineCandidateConstructors(AutowiredAnnotationBeanPostProcessor.java:267)
    ... 76 common frames omitted
Caused by: java.lang.NoClassDefFoundError: io/gravitee/definition/model/Path
    at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
    at java.base/java.lang.Class.privateGetDeclaredMethods(Unknown Source)
    at java.base/java.lang.Class.getDeclaredMethods(Unknown Source)
    at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:467)
    ... 78 common frames omitted
Caused by: java.lang.ClassNotFoundException: io.gravitee.definition.model.Path
    at java.base/java.net.URLClassLoader.findClass(Unknown Source)
    at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
    at java.base/java.net.FactoryURLClassLoader.loadClass(Unknown Source)
    at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
    ... 82 common frames omitted
05:22:11.154 [graviteeio-node] [] ERROR i.g.n.p.s.h.ServicePluginHandler - Unexpected error while creating ingress
java.lang.IllegalStateException: org.springframework.context.annotation.AnnotationConfigApplicationContext@751b901a has not been refreshed yet
    at org.springframework.context.support.AbstractApplicationContext.assertBeanFactoryActive(AbstractApplicationContext.java:1141)
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1171)
    at io.gravitee.plugin.core.api.AbstractSpringPluginHandler.handle(AbstractSpringPluginHandler.java:34)
    at io.gravitee.plugin.core.api.AbstractPluginHandler.handle(AbstractPluginHandler.java:65)
    at io.gravitee.plugin.core.internal.PluginEventListener.lambda$deployPlugin$8(PluginEventListener.java:125)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source)
    at java.base/java.util.stream.ReferencePipeline$2$1.accept(Unknown Source)
    at java.base/java.util.Iterator.forEachRemaining(Unknown Source)
    at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Unknown Source)
    at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
    at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(Unknown Source)
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(Unknown Source)
    at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
    at java.base/java.util.stream.ReferencePipeline.forEach(Unknown Source)
    at io.gravitee.plugin.core.internal.PluginEventListener.deployPlugin(PluginEventListener.java:123)
    at io.gravitee.plugin.core.internal.PluginEventListener.lambda$deployPlugins$5(PluginEventListener.java:108)
    at java.base/java.util.ArrayList.forEach(Unknown Source)
    at io.gravitee.plugin.core.internal.PluginEventListener.deployPlugins(PluginEventListener.java:108)
    at io.gravitee.plugin.core.internal.PluginEventListener.onEvent(PluginEventListener.java:67)
    at io.gravitee.common.event.impl.EventManagerImpl.publishEvent(EventManagerImpl.java:43)
    at io.gravitee.common.event.impl.EventManagerImpl.publishEvent(EventManagerImpl.java:35)
    at io.gravitee.plugin.core.internal.PluginRegistryImpl.init(PluginRegistryImpl.java:115)
    at io.gravitee.plugin.core.internal.PluginRegistryImpl.doStart(PluginRegistryImpl.java:91)
    at io.gravitee.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:32)
    at io.gravitee.node.container.AbstractNode.startComponents(AbstractNode.java:202)
    at io.gravitee.node.container.AbstractNode.doStart(AbstractNode.java:76)
    at io.gravitee.gateway.standalone.node.GatewayNode.doStart(GatewayNode.java:78)
    at io.gravitee.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:32)
    at io.gravitee.node.container.AbstractContainer.doStart(AbstractContainer.java:106)
    at io.gravitee.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:32)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.base/java.lang.reflect.Method.invoke(Unknown Source)
    at io.gravitee.gateway.standalone.boostrap.Bootstrap.start(Bootstrap.java:154)

Here is the values.yaml file used for the Gravitee gateway installation -

gateway:
  enabled: true
  type: Deployment
  name: gateway
  reloadOnConfigChange: true
  autoscaling:
    enabled: false
  replicaCount: 1
  ingress:
    enabled: true
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/ssl-redirect: "false"
    hosts:
      - localhost
    ingressClassName: ""
    path: /gateway
  resources:
    limits:
      cpu: 500m
      memory: 512Mi
    requests:
      cpu: 200m
      memory: 256Mi
  services:
    kubeController:
      enabled: true
      serviceAccount: "gravitee-gateway-sa"
  additionalPlugins:
    - https://download.gravitee.io/graviteeio-apim/plugins/services/gravitee-kubernetes-controller/gravitee-kubernetes-controller-0.1.0.zip
mongo:
  uri: mongodb://root:r00t@mongodb:27017/admin?connectTimeoutMS=30000
  auth:
    enabled: true
    source: admin
    username: root
    password: r00t
  rsEnabled: false
es:
  enabled: true
  endpoints:
    - http://elastic-elasticsearch-coordinating-only.gravitee-apim.svc.cluster.local:9200
elasticsearch:
  enabled: false
management:
  type: mongodb
ratelimit:
  type: mongodb
api:
  enabled: false
portal:
  enabled: false
ui:
  enabled: false
smtp:
  enabled: false
notifiers:
  smtp:
    enabled: false
sanjimoh commented 2 years ago

Complete gateway pod logs attached - graviteeio-apim3x-gateway-844846d49f-jpnv4.log

sanjimoh commented 2 years ago

Hi, I would really appreciate if someone can help me here. Thanks!

sanjimoh commented 2 years ago

Looking at the error, it appears to me, is there some version mismatch? Something got dropped causing a backward incompatible change or similar?

Caused by: java.lang.ClassNotFoundException: io.gravitee.definition.model.Path
    at java.base/java.net.URLClassLoader.findClass(Unknown Source)
    at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
    at java.base/java.net.FactoryURLClassLoader.loadClass(Unknown Source)
    at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
    ... 82 common frames omitted

I took a look at the gravitee-api-management codebase & I do not any Path definition -

https://github.com/gravitee-io/gravitee-api-management/tree/3.15.10/gravitee-apim-definition/gravitee-apim-definition-model/src/main/java/io/gravitee/definition/model