search

spring-cloud / spring-cloud-netflix

Integration with Netflix OSS components
http://cloud.spring.io/spring-cloud-netflix/
Apache License 2.0
4.87k stars 2.44k forks source link

No route to host executing #769

Closed Dreampie closed 8 years ago

Dreampie commented 8 years ago

link https://github.com/Netflix/feign/issues/314

@FeignClient("user-service-provider")
public interface UserService {
}

image

image

image

user-service-provider always success,sometimes user-api-provider can't read from user-service-provider,other times it's ok.

image

image

error:

2016-01-14 08:32:36.782 ERROR 1 --- [ XNIO-2 task-16] io.undertow.request                      : UT005023: Exception handling request to /users

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is feign.RetryableException: No route to host executing GET http://user-service-provider/users
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:978)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:857)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:130)
    at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:291)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:102)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)
    at org.springframework.boot.actuate.autoconfigure.MetricFilterAutoConfiguration$MetricsFilter.doFilterInternal(MetricFilterAutoConfiguration.java:90)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)
    at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85)
    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:61)
    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:56)
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:45)
    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:63)
    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)
    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70)
    at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261)
    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:247)
    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:76)
    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:166)
    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:197)
    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:765)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: feign.RetryableException: No route to host executing GET http://user-service-provider/users
    at feign.FeignException.errorExecuting(FeignException.java:56)
    at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:97)
    at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:71)
    at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:94)
    at com.sun.proxy.$Proxy85.findAll(Unknown Source)
    at tv.acfun.cloud.api.user.controller.UserController.findAll(UserController.java:24)
    at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:776)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:705)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
    ... 48 common frames omitted
Caused by: java.net.NoRouteToHostException: No route to host
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
    at sun.net.www.http.HttpClient.New(HttpClient.java:308)
    at sun.net.www.http.HttpClient.New(HttpClient.java:326)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
    at feign.Client$Default.convertResponse(Client.java:143)
    at feign.Client$Default.execute(Client.java:70)
    at feign.ribbon.LBClient.execute(LBClient.java:69)
    at feign.ribbon.LBClient.execute(LBClient.java:38)
    at com.netflix.client.AbstractLoadBalancerAwareClient$1.call(AbstractLoadBalancerAwareClient.java:109)
    at com.netflix.loadbalancer.reactive.LoadBalancerCommand$3$1.call(LoadBalancerCommand.java:303)
    at com.netflix.loadbalancer.reactive.LoadBalancerCommand$3$1.call(LoadBalancerCommand.java:287)
    at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55)
    at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:41)
    at rx.internal.util.ScalarSynchronousObservable$1.call(ScalarSynchronousObservable.java:30)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable.unsafeSubscribe(Observable.java:7466)
    at rx.internal.operators.OperatorConcat$ConcatSubscriber.subscribeNext(OperatorConcat.java:176)
    at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:141)
    at rx.internal.operators.OperatorConcat$ConcatSubscriber.onNext(OperatorConcat.java:78)
    at rx.internal.operators.OperatorMap$1.onNext(OperatorMap.java:55)
    at com.netflix.loadbalancer.reactive.LoadBalancerCommand$1.call(LoadBalancerCommand.java:185)
    at com.netflix.loadbalancer.reactive.LoadBalancerCommand$1.call(LoadBalancerCommand.java:180)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable.subscribe(Observable.java:7556)
    at rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:442)
    at rx.observables.BlockingObservable.single(BlockingObservable.java:341)
    at com.netflix.client.AbstractLoadBalancerAwareClient.executeWithLoadBalancer(AbstractLoadBalancerAwareClient.java:102)
    at feign.ribbon.RibbonClient.execute(RibbonClient.java:69)
    at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:92)
    ... 64 common frames omitted
dsyer commented 8 years ago

I don't see how it can sometimes succeed and sometimes not in the same app ,but you have 4 instances of the user-api-provider, which I gather is the caller. Maybe one of them isn't configured correctly. We'll need a much simpler scenario to reproduce if there really is an issue in Spring Cloud.

Dreampie commented 8 years ago

machine 43,44 start in docker sometimes failure,100 not in docker always success.why?

I use argument EUREKA_INSTANCE_IP to set eureka client ip with host ip

eureka:
  password: eureka
  client:
    healthcheck:
      enabled: true
    serviceUrl:
      defaultZone: ${EUREKA_SERVER_URL_1:http://user:${eureka.password}@eureka1.cloud.acfun.tv:8761/eureka/},${EUREKA_SERVER_URL_2:http://user:${eureka.password}@eureka2.cloud.acfun.tv:8761/eureka/}
  instance:
    preferIpAddress: true
    ipAddress: ${EUREKA_INSTANCE_IP}
    leaseRenewalIntervalInSeconds: 10
    statusPageUrlPath: /info
    healthCheckUrlPath: /health
    metadataMap:
      instanceId: ${EUREKA_INSTANCE_ID:${spring.application.name}:${spring.application.instance_id:${server.port:8080}}}
dsyer commented 8 years ago

Docker networking is a nightmare, but it isn't really anything to do with Spring Cloud. Maybe you can distil it down to a simpler problem (a single discovery client that can't contact Eureka)?

Dreampie commented 8 years ago

but eureka show me all up.how can i found any client was down?

dsyer commented 8 years ago

I don't think eureka is the problem, but if any client is down ribbon should move on to the next one (if you enable retry on that client). The real problem is probably the network.

spencergibb commented 8 years ago

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

zios07 commented 6 years ago

Any solution to that problem ?

ZhaoDaMing commented 2 years ago

I have the same problem. This error occurs in FeignClient when rolling Deployment using Deployment in K8S. I suspect the root cause is that FeignClient keeps an old list of service providers and the Ribbon cannot move correctly to the next node if one node has been destroyed

arundotin commented 2 years ago

@spencergibb - any solution to this problem ? facing a similar issue in our prod environment. Only restart is helping us