How to persist the plugins directory in docker images?

cacidavidschermers commented 7 years ago

We use the gravitee.io docker images in our openshift platform. We wrote a own plugin which must be uploaded in the gateway and api-management docker images. I can upload the plugin to the running pods but the pods needs a restart to be activated. We've added persistent storage to the pods but the api-management pod is in error after adding the storage.

Expected Behavior

Pod has persistent storage, expect to startup the gateway and api-management pods correctly

Current Behavior

Pod has persistent storage, gateway pod start correctly, the api-management pod is not starting because of this exception:

Factory method 'vertx' threw exception; nested exception is java.lang.IllegalStateException: Failed to create cache dir

How can I put the plugin to the pods and keep the pods persistent to be able to restart the pods and activate the plugins?

Your Environment

OpenShift platform where the docker images are added

NicolasGeraud commented 7 years ago

hi @cacidavidschermers ,

2 questions :

why not having build your own docker images (with our as root) and copy your plugin into the plugin folder ? Something like :

FROM graviteeio/management-api:latest
COPY my-plugin.zip plugins/

can you share the full stacktrace. On startup, Gravitee.io unzip plugins in a cache folder. Perhaps you have set readonly rights to the storage.

cacidavidschermers commented 7 years ago

Hello Nicolas,

Creating our own docker images is an option. If that's not necessary I would like to prevent that.

The storage has read/write rights. The full stacktrace:

07:34:05.689 [gravitee] WARN o.s.c.a.AnnotationConfigApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'nodeHttpServer': Unsatisfied dependency expressed through field 'httpServer'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'vertxNodeHttpServer' defined in io.gravitee.management.services.http.spring.HttpServerSpringConfiguration: Unsatisfied dependency expressed through method 'httpServer' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'vertx' defined in io.gravitee.management.services.http.spring.HttpServerSpringConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.vertx.core.Vertx]: Factory method 'vertx' threw exception; nested exception is java.lang.IllegalStateException: Failed to create cache dir org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'nodeHttpServer': Unsatisfied dependency expressed through field 'httpServer'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'vertxNodeHttpServer' defined in io.gravitee.management.services.http.spring.HttpServerSpringConfiguration: Unsatisfied dependency expressed through method 'httpServer' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'vertx' defined in io.gravitee.management.services.http.spring.HttpServerSpringConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.vertx.core.Vertx]: Factory method 'vertx' threw exception; nested exception is java.lang.IllegalStateException: Failed to create cache dir at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:569) at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:349) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1219) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:543) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:751) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:861) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:541) at io.gravitee.management.standalone.Container.initializeContext(Container.java:94) at io.gravitee.management.standalone.Container.initialize(Container.java:53) at io.gravitee.management.standalone.Container.(Container.java:44) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at io.gravitee.management.standalone.boostrap.Bootstrap.init(Bootstrap.java:50) at io.gravitee.management.standalone.boostrap.Bootstrap.start(Bootstrap.java:153) at io.gravitee.management.standalone.boostrap.Bootstrap.main(Bootstrap.java:166) Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'vertxNodeHttpServer' defined in io.gravitee.management.services.http.spring.HttpServerSpringConfiguration: Unsatisfied dependency expressed through method 'httpServer' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'vertx' defined in io.gravitee.management.services.http.spring.HttpServerSpringConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.vertx.core.Vertx]: Factory method 'vertx' threw exception; nested exception is java.lang.IllegalStateException: Failed to create cache dir at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:749) at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:467) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1128) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1023) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:510) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:207) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1128) at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1056) at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:566) ... 23 more Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'vertx' defined in io.gravitee.management.services.http.spring.HttpServerSpringConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.vertx.core.Vertx]: Factory method 'vertx' threw exception; nested exception is java.lang.IllegalStateException: Failed to create cache dir at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:599) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1128) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1023) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:510) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:207) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1128) at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1056) at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:835) at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:741) ... 36 more Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.vertx.core.Vertx]: Factory method 'vertx' threw exception; nested exception is java.lang.IllegalStateException: Failed to create cache dir at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189) at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588) ... 49 more Caused by: java.lang.IllegalStateException: Failed to create cache dir at io.vertx.core.impl.FileResolver.setupCacheDir(FileResolver.java:293) at io.vertx.core.impl.FileResolver.(FileResolver.java:86) at io.vertx.core.impl.VertxImpl.(VertxImpl.java:175) at io.vertx.core.impl.VertxImpl.(VertxImpl.java:142) at io.vertx.core.impl.VertxImpl.(VertxImpl.java:138) at io.vertx.core.impl.VertxFactoryImpl.vertx(VertxFactoryImpl.java:34) at io.vertx.core.Vertx.vertx(Vertx.java:80) at io.gravitee.management.services.http.spring.HttpServerSpringConfiguration.vertx(HttpServerSpringConfiguration.java:67) at io.gravitee.management.services.http.spring.HttpServerSpringConfiguration$$EnhancerBySpringCGLIB$$6ff2d675.CGLIB$vertx$5() at io.gravitee.management.services.http.spring.HttpServerSpringConfiguration$$EnhancerBySpringCGLIB$$6ff2d675$$FastClassBySpringCGLIB$$9e8228e2.invoke() at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:356) at io.gravitee.management.services.http.spring.HttpServerSpringConfiguration$$EnhancerBySpringCGLIB$$6ff2d675.vertx() at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162) ... 50 more

NicolasGeraud commented 7 years ago

Vertx cannot create is temporary cache dir (https://github.com/eclipse/vert.x/blob/dae6d1635cba75dd8c37776cac355d7175cb8cdd/src/main/java/io/vertx/core/impl/FileResolver.java#L299).

I don't know how openshift works, but it seems to be a misconfiguration in the storage settings.Is there any difference between the configuration of the gateway and the management-api ?

brasseld commented 7 years ago

We have to add this jvm options in launcher : -Dvertx.disableFileCaching=true

NicolasGeraud commented 7 years ago

Why ? Vertx should be able to create its cache directory. Am I wrong ?

brasseld commented 7 years ago

We do not need it. Please have a look to gateway launcher

gravitee-io / issues