search

fabric8io / fabric8-ipaas

This repository contains the iPaaS related apps that can be run as part of the fabric8 platform on any OpenShift v3 and Kubernetes environment
14 stars 33 forks source link

[apiman] apiman breaks during startup if ElasticSearch is not available #181

Closed jpechane closed 8 years ago

jpechane commented 8 years ago

OpenShift does not guarantee the order of pods during the start. If ElasticSearch is initialized AFTER apiman manager then the manager is not available and has to be restarted. As OpenShift is elastic environment it should be able to survive such situation and be able to work when ES is available.

707:17:51,108  WARN FAILED o.e.j.s.ServletContextHandler@29d070c7{/apiman,null,STARTING}: java.lang.RuntimeException: org.apache.http.NoHttpResponseException: elasticsearch-v1.jpechane.svc.cluster.local:9200 failed to respond
java.lang.RuntimeException: org.apache.http.NoHttpResponseException: elasticsearch-v1.jpechane.svc.cluster.local:9200 failed to respond
    at io.apiman.manager.api.es.EsStorage.initialize(EsStorage.java:164)
    at io.apiman.manager.api.micro.ManagerApiMicroServiceCdiFactory.initES(ManagerApiMicroServiceCdiFactory.java:290)
    at io.apiman.manager.api.micro.ManagerApiMicroServiceCdiFactory.provideStorageQuery(ManagerApiMicroServiceCdiFactory.java:139)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:88)
    at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:78)
    at org.jboss.weld.injection.producer.ProducerMethodProducer.produce(ProducerMethodProducer.java:98)
    at org.jboss.weld.injection.producer.AbstractMemberProducer.produce(AbstractMemberProducer.java:161)
    at org.jboss.weld.bean.AbstractProducerBean.create(AbstractProducerBean.java:181)
    at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:96)
    at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:101)
    at org.jboss.weld.bean.ContextualInstanceStrategy$ApplicationScopedContextualInstanceStrategy.get(ContextualInstanceStrategy.java:141)
    at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
    at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:99)
    at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:99)
    at org.jboss.weld.proxies.IStorageQuery$825249753$Proxy$_$$_WeldClientProxy.listPolicyDefinitions(Unknown Source)
    at io.fabric8.apiman.BootstrapFilter.loadDefaultPolicies(BootstrapFilter.java:85)
    at io.fabric8.apiman.BootstrapFilter.init(BootstrapFilter.java:239)
    at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:138)
    at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:852)
    at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
    at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
    at org.eclipse.jetty.server.Server.start(Server.java:387)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
    at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
    at org.eclipse.jetty.server.Server.doStart(Server.java:354)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at io.apiman.manager.api.micro.ManagerApiMicroService.start(ManagerApiMicroService.java:81)
    at io.fabric8.apiman.ApimanStarter.main(ApimanStarter.java:42)
jpechane commented 8 years ago

Or the problem has to be solved with liveness/readiness probe

KurtStam commented 8 years ago

I'm waiting for apiman-1.2.2, https://issues.jboss.org/browse/APIMAN-914

KurtStam commented 8 years ago

https://github.com/fabric8io/fabric8-ipaas/pull/201