search

4finance / micro-infra-spring

Repository containing default microservice infrastructure set up using Spring configuration
Apache License 2.0
203 stars 49 forks source link

stub-runner hangs when artifacts reposiotry is not responding #230

Open nurkiewicz opened 9 years ago

nurkiewicz commented 9 years ago

Issue by szpak Friday Dec 19, 2014 at 17:34 GMT Originally opened as https://github.com/4finance/stub-runner/issues/23

For example when a server is not accessible without a company VPN. It just hangs for a few minute (possible longer - not tested). In the result the end application startup hangs without any sign what is going on.

A fragment of the long thread dump:

"main" prio=10 tid=0x00007f997c024800 nid=0x3898 runnable [0x00007f9985840000]
   java.lang.Thread.State: RUNNABLE
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    - locked <0x00000000ed538c48> (a java.net.SocksSocketImpl)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:579)
    at java.net.Socket.connect(Socket.java:528)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
    - locked <0x00000000ed538bc0> (a sun.net.www.http.HttpClient)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
    at sun.net.www.http.HttpClient.New(HttpClient.java:308)
    at sun.net.www.http.HttpClient.New(HttpClient.java:326)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:996)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:932)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:850)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1300)
    - locked <0x00000000ed538108> (a sun.net.www.protocol.http.HttpURLConnection)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
    at org.apache.ivy.util.url.BasicURLHandler.checkStatusCode(BasicURLHandler.java:130)
    at org.apache.ivy.util.url.BasicURLHandler.openStream(BasicURLHandler.java:165)
    at org.apache.ivy.util.url.ApacheURLLister.retrieveListing(ApacheURLLister.java:112)
    at org.apache.ivy.util.url.ApacheURLLister.listAll(ApacheURLLister.java:57)
    at org.apache.ivy.plugins.repository.url.URLRepository.list(URLRepository.java:100)
    at org.apache.ivy.plugins.resolver.util.ResolverHelper.listAll(ResolverHelper.java:108)
    at org.apache.ivy.plugins.resolver.util.ResolverHelper.listTokenValues(ResolverHelper.java:57)
    at org.apache.ivy.plugins.resolver.util.ResolverHelper.findAll(ResolverHelper.java:144)
    at org.apache.ivy.plugins.resolver.RepositoryResolver.listResources(RepositoryResolver.java:185)
    at org.apache.ivy.plugins.resolver.IBiblioResolver.listResources(IBiblioResolver.java:428)
    at org.apache.ivy.plugins.resolver.RepositoryResolver.findDynamicResourceUsingPattern(RepositoryResolver.java:149)
    at org.apache.ivy.plugins.resolver.RepositoryResolver.findResourceUsingPattern(RepositoryResolver.java:132)
    at org.apache.ivy.plugins.resolver.AbstractPatternsBasedResolver.findResourceUsingPatterns(AbstractPatternsBasedResolver.java:96)
    at org.apache.ivy.plugins.resolver.IBiblioResolver.findIvyFileRef(IBiblioResolver.java:102)
    at org.apache.ivy.plugins.resolver.BasicResolver.getDependency(BasicResolver.java:228)
    at org.apache.ivy.plugins.resolver.IBiblioResolver.getDependency(IBiblioResolver.java:512)
    at org.apache.ivy.plugins.resolver.ChainResolver.getDependency(ChainResolver.java:104)
    at org.apache.ivy.core.resolve.IvyNode.loadData(IvyNode.java:169)
    at org.apache.ivy.core.resolve.VisitNode.loadData(VisitNode.java:292)
    at org.apache.ivy.core.resolve.ResolveEngine.fetchDependencies(ResolveEngine.java:714)
    at org.apache.ivy.core.resolve.ResolveEngine.doFetchDependencies(ResolveEngine.java:799)
    at org.apache.ivy.core.resolve.ResolveEngine.fetchDependencies(ResolveEngine.java:722)
    at org.apache.ivy.core.resolve.ResolveEngine.getDependencies(ResolveEngine.java:594)
    at org.apache.ivy.core.resolve.ResolveEngine.resolve(ResolveEngine.java:234)
    at org.apache.ivy.Ivy.resolve(Ivy.java:517)
    at org.apache.ivy.Ivy$resolve$0.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
    at groovy.grape.GrapeIvy.getDependencies(GrapeIvy.groovy:406)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSite.invoke(PogoMetaMethodSite.java:166)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:56)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
    at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:565)
    at groovy.grape.GrapeIvy$resolve$2.callCurrent(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:153)
    at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:530)
    at groovy.grape.Grape.resolve(Grape.java:198)
    at groovy.grape.Grape.resolve(Grape.java:184)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
    at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.invoke(StaticMetaMethodSite.java:43)
    at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.callStatic(StaticMetaMethodSite.java:99)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:53)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:157)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:169)
    at com.ofg.stub.config.StubRunnerConfiguration$DependencyResolver.resolveDependencyLocation(StubRunnerConfiguration.groovy:211)
    at com.ofg.stub.config.StubRunnerConfiguration$DependencyResolver$resolveDependencyLocation.callCurrent(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
    at com.ofg.stub.config.StubRunnerConfiguration$RemoteDependencyResolver.doResolveRemoteDependency(StubRunnerConfiguration.groovy:163)
    at com.ofg.stub.config.StubRunnerConfiguration$RemoteDependencyResolver.this$3$doResolveRemoteDependency(StubRunnerConfiguration.groovy)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:207)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:56)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:145)
    at com.ofg.stub.config.StubRunnerConfiguration$RemoteDependencyResolver.resolveDependency(StubRunnerConfiguration.groovy:137)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:207)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:68)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
    at com.ofg.stub.config.StubRunnerConfiguration.findGrabbedStubJars(StubRunnerConfiguration.groovy:111)
    at com.ofg.stub.config.StubRunnerConfiguration.batchStubRunner(StubRunnerConfiguration.groovy:87)
    at com.ofg.stub.config.StubRunnerConfiguration$$EnhancerBySpringCGLIB$$36632f68.CGLIB$batchStubRunner$23(<generated>)
    at com.ofg.stub.config.StubRunnerConfiguration$$EnhancerBySpringCGLIB$$36632f68$$FastClassBySpringCGLIB$$2fb274e.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:312)
    at com.ofg.stub.config.StubRunnerConfiguration$$EnhancerBySpringCGLIB$$36632f68.batchStubRunner(<generated>)
(...)
nurkiewicz commented 9 years ago

Comment by marcingrzejszczak Friday Dec 19, 2014 at 17:49 GMT

From ivy's perspective we might have a problem:

https://issues.apache.org/jira/browse/IVY-735

Maybe... but this is a little bit insane ;) ... we can run grabbing in a Thread that would have certain time to return a result - and if takes too long we would try to interrupt the thread. Don't know if that would actualy be feasible and also sounds really bad :P

nurkiewicz commented 9 years ago

Comment by nurkiewicz Thursday Jan 08, 2015 at 20:31 GMT

Let's make it even more insane. I suggest starting in-memory ZooKeeper first, but move downloading stubs and starting WireMock servers into a background thread. This way we don't slow down application startup and after all, application shouldn't assume all dependencies are available right from the beginning.

Of course there is a problem with error handling. Should we ignore problems while downloading stubs (timeouts, other exceptions) or shut down application in some unexpected moment?

nurkiewicz commented 9 years ago

Comment by marcingrzejszczak Sunday Jan 11, 2015 at 16:09 GMT

I see only one issue here - what if the tests execute themselves faster then the stubs are downloaded? Your tests will fail even though they shouldn't.

nurkiewicz commented 9 years ago

Comment by nurkiewicz Sunday Jan 11, 2015 at 18:19 GMT

Not necessarily. During tests (test profile?) this feature can be disabled by default. Moreover during integration tests we can somehow ask application from outside whether stubs are ready. This can be done by calling /collaborators. Of course there will be a flag (e.g. stubrunner.async-startup) at user disposal.