search

spring-cloud / spring-cloud-openfeign

Support for using OpenFeign in Spring Cloud apps
Apache License 2.0
1.21k stars 785 forks source link

[Help Wanted] "Load balancer does not contain an instance for the service" - Use Feign Client in Application Ready Event #459

Closed mikemikezhu closed 11 months ago

mikemikezhu commented 3 years ago

Project Requirement:

I want to send an HTTP request using Feign Client from ack-service to notification-service when ack-service get initialized (e.g. Application is ready) in the Spring Cloud application so that notification-service has the configuration information about ack-service

Description:

GIVEN we have Eureka server: "discovery-service" and Feign Clients: "ack-service", "notification-service" GIVEN "ack-service" and "notification-service" have registered into "discovery-service" GIVEN "ack-service" listens to ApplicationReadyEvent WHEN "ack-service" sends data to "notification-service" using Feign Clients when application is ready THEN we have Load balancer does not contain an instance for the service error message

Details:

  1. NotificationClient in ack-service, where we defined the Feign Client

    @FeignClient("notification-service")
public interface NotificationClient {

@PostMapping("/add_client_info")
ResponseEntity<String> addClientInfo(@RequestBody IntraNotificationClientInfo clientInfo);
}

  2. IntraNotificationController in notification-service, where we implemented the Feign Client

    @RestController
public class IntraNotificationController {

private final CoreNotificationCenter notificationCenter;

@Autowired
public IntraNotificationController(final CoreNotificationCenter notificationCenter) {
    this.notificationCenter = notificationCenter;
}

@PostMapping("/add_client_info")
public ResponseEntity<String> addClientInfo(@RequestBody IntraNotificationClientInfo clientInfo) {

    try {
        // Add client info
        notificationCenter.addClientInfo(clientInfo);
    } catch (IllegalStateException e) {
        e.printStackTrace();
        return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(e.getMessage());
    }

    return ResponseEntity.ok().build();
}
}

  3. CoreAckHandler in ack-service, which listens to ApplicationReadyEvent and invoke the Feign Client

    @Component
public class CoreAckHandler {

private static final Logger LOGGER = LoggerFactory.getLogger(CoreAckHandler.class);

private final NotificationClient notificationClient;

@Autowired
public CoreAckHandler(final NotificationClient notificationClient) {
    this.notificationClient = notificationClient;
}

@EventListener(ApplicationReadyEvent.class)
public void onApplicationReady() {

    LOGGER.info("******************** On application ready");

    // Add client info to core notification center
    final IntraNotificationClientInfo clientInfo = new IntraNotificationClientInfo(
            AckConstants.ACK_CLIENT_NAME, AckConstants.NOTIFICATION_TYPE_ON_CLIENT_ACK);
    notificationClient.addClientInfo(clientInfo); // ==================> "Load balancer does not contain an instance for the service" error
}
}

Then we have the following error:

feign.FeignException$ServiceUnavailable: [503] during [POST] to [http://notification-service/add_client_info] [NotificationClient#addClientInfo(IntraNotificationClientInfo)]: [Load balancer does not contain an instance for the service notification-service]
    at feign.FeignException.serverErrorStatus(FeignException.java:237) ~[feign-core-10.10.1.jar:na]
    at feign.FeignException.errorStatus(FeignException.java:180) ~[feign-core-10.10.1.jar:na]
    at feign.FeignException.errorStatus(FeignException.java:169) ~[feign-core-10.10.1.jar:na]
    at feign.codec.ErrorDecoder$Default.decode(ErrorDecoder.java:92) ~[feign-core-10.10.1.jar:na]
    at feign.AsyncResponseHandler.handleResponse(AsyncResponseHandler.java:96) ~[feign-core-10.10.1.jar:na]
    at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:138) ~[feign-core-10.10.1.jar:na]
    at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:89) ~[feign-core-10.10.1.jar:na]
    at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:100) ~[feign-core-10.10.1.jar:na]
    at com.sun.proxy.$Proxy79.addClientInfo(Unknown Source) ~[na:na]
    at com.myproject.ack.handler.CoreAckHandler.onApplicationReady(CoreAckHandler.java:55) ~[classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_181]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_181]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_181]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_181]
    at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:312) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:197) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:160) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:426) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:383) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.boot.context.event.EventPublishingRunListener.running(EventPublishingRunListener.java:111) ~[spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplicationRunListeners.lambda$running$6(SpringApplicationRunListeners.java:79) ~[spring-boot-2.4.2.jar:2.4.2]
    at java.util.ArrayList.forEach(ArrayList.java:1257) ~[na:1.8.0_181]
    at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:117) ~[spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:111) ~[spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplicationRunListeners.running(SpringApplicationRunListeners.java:79) ~[spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:341) [spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1311) [spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1300) [spring-boot-2.4.2.jar:2.4.2]
    at com.myproject.ack.AckApplication.main(AckApplication.java:14) [classes/:na]

Note:

  1. When using Feign Client inside @PostConstruct, and invoked after dependency injection is done, we have the error Load balancer does not contain an instance for the service (I think it is because the Eureka client is not registered yet at this time)
  2. When using Feign Client inside @EventListener(ApplicationReadyEvent.class), sometimes we have the error Load balancer does not contain an instance for the service, while sometimes there is working fine
  3. When using Feign Client inside @RestController to handle HTTP requests sent from clients, then this is working fine

My questions are:

  1. How to use Feign Client to send notification from one micro-service to another when the app gets initialized (e.g. Application is ready)?
  2. Is there any event listener in Spring Cloud which can indicate the Eureka and Feign client is fully registered?

Thank you so much for your help!

P.s. Full Log:

/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/bin/java -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:53072,suspend=y,server=n -XX:TieredStopAtLevel=1 -noverify -Dspring.output.ansi.enabled=always -javaagent:/Users/Mike/Library/Caches/JetBrains/IntelliJIdea2020.3/captureAgent/debugger-agent.jar -Dcom.sun.management.jmxremote -Dspring.jmx.enabled=true -Dspring.liveBeansView.mbeanDomain -Dspring.application.admin.enabled=true -Dfile.encoding=UTF-8 -classpath /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/lib/tools.jar:/Users/Mike/Documents/OnlineCourse/SCI/Projects/EnerSave/EnerSave/federated_learning_enersave/EnerSave-Cloud/ack-service/target/classes:/Users/Mike/.m2/repository/org/springframework/boot/spring-boot-starter-web/2.4.2/spring-boot-starter-web-2.4.2.jar:/Users/Mike/.m2/repository/org/springframework/boot/spring-boot-starter/2.4.2/spring-boot-starter-2.4.2.jar:/Users/Mike/.m2/repository/org/springframework/boot/spring-boot/2.4.2/spring-boot-2.4.2.jar:/Users/Mike/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/2.4.2/spring-boot-autoconfigure-2.4.2.jar:/Users/Mike/.m2/repository/org/springframework/boot/spring-boot-starter-logging/2.4.2/spring-boot-starter-logging-2.4.2.jar:/Users/Mike/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar:/Users/Mike/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar:/Users/Mike/.m2/repository/org/apache/logging/log4j/log4j-to-slf4j/2.13.3/log4j-to-slf4j-2.13.3.jar:/Users/Mike/.m2/repository/org/apache/logging/log4j/log4j-api/2.13.3/log4j-api-2.13.3.jar:/Users/Mike/.m2/repository/org/slf4j/jul-to-slf4j/1.7.30/jul-to-slf4j-1.7.30.jar:/Users/Mike/.m2/repository/jakarta/annotation/jakarta.annotation-api/1.3.5/jakarta.annotation-api-1.3.5.jar:/Users/Mike/.m2/repository/org/yaml/snakeyaml/1.27/snakeyaml-1.27.jar:/Users/Mike/.m2/repository/org/springframework/boot/spring-boot-starter-json/2.4.2/spring-boot-starter-json-2.4.2.jar:/Users/Mike/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.11.4/jackson-databind-2.11.4.jar:/Users/Mike/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.11.4/jackson-datatype-jdk8-2.11.4.jar:/Users/Mike/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.11.4/jackson-datatype-jsr310-2.11.4.jar:/Users/Mike/.m2/repository/com/fasterxml/jackson/module/jackson-module-parameter-names/2.11.4/jackson-module-parameter-names-2.11.4.jar:/Users/Mike/.m2/repository/org/springframework/boot/spring-boot-starter-tomcat/2.4.2/spring-boot-starter-tomcat-2.4.2.jar:/Users/Mike/.m2/repository/org/apache/tomcat/embed/tomcat-embed-core/9.0.41/tomcat-embed-core-9.0.41.jar:/Users/Mike/.m2/repository/org/glassfish/jakarta.el/3.0.3/jakarta.el-3.0.3.jar:/Users/Mike/.m2/repository/org/apache/tomcat/embed/tomcat-embed-websocket/9.0.41/tomcat-embed-websocket-9.0.41.jar:/Users/Mike/.m2/repository/org/springframework/spring-web/5.3.3/spring-web-5.3.3.jar:/Users/Mike/.m2/repository/org/springframework/spring-beans/5.3.3/spring-beans-5.3.3.jar:/Users/Mike/.m2/repository/org/springframework/spring-webmvc/5.3.3/spring-webmvc-5.3.3.jar:/Users/Mike/.m2/repository/org/springframework/spring-aop/5.3.3/spring-aop-5.3.3.jar:/Users/Mike/.m2/repository/org/springframework/spring-context/5.3.3/spring-context-5.3.3.jar:/Users/Mike/.m2/repository/org/springframework/spring-expression/5.3.3/spring-expression-5.3.3.jar:/Users/Mike/.m2/repository/org/springframework/cloud/spring-cloud-starter-netflix-eureka-client/3.0.0/spring-cloud-starter-netflix-eureka-client-3.0.0.jar:/Users/Mike/.m2/repository/org/springframework/cloud/spring-cloud-starter/3.0.0/spring-cloud-starter-3.0.0.jar:/Users/Mike/.m2/repository/org/springframework/cloud/spring-cloud-context/3.0.0/spring-cloud-context-3.0.0.jar:/Users/Mike/.m2/repository/org/springframework/security/spring-security-rsa/1.0.9.RELEASE/spring-security-rsa-1.0.9.RELEASE.jar:/Users/Mike/.m2/repository/org/bouncycastle/bcpkix-jdk15on/1.64/bcpkix-jdk15on-1.64.jar:/Users/Mike/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.64/bcprov-jdk15on-1.64.jar:/Users/Mike/.m2/repository/org/springframework/cloud/spring-cloud-netflix-eureka-client/3.0.0/spring-cloud-netflix-eureka-client-3.0.0.jar:/Users/Mike/.m2/repository/com/netflix/eureka/eureka-client/1.10.10/eureka-client-1.10.10.jar:/Users/Mike/.m2/repository/com/netflix/netflix-commons/netflix-eventbus/0.3.0/netflix-eventbus-0.3.0.jar:/Users/Mike/.m2/repository/com/netflix/netflix-commons/netflix-infix/0.3.0/netflix-infix-0.3.0.jar:/Users/Mike/.m2/repository/commons-jxpath/commons-jxpath/1.3/commons-jxpath-1.3.jar:/Users/Mike/.m2/repository/joda-time/joda-time/2.3/joda-time-2.3.jar:/Users/Mike/.m2/repository/org/antlr/antlr-runtime/3.4/antlr-runtime-3.4.jar:/Users/Mike/.m2/repository/org/antlr/stringtemplate/3.2.1/stringtemplate-3.2.1.jar:/Users/Mike/.m2/repository/antlr/antlr/2.7.7/antlr-2.7.7.jar:/Users/Mike/.m2/repository/org/apache/commons/commons-math/2.2/commons-math-2.2.jar:/Users/Mike/.m2/repository/javax/ws/rs/jsr311-api/1.1.1/jsr311-api-1.1.1.jar:/Users/Mike/.m2/repository/com/netflix/servo/servo-core/0.12.21/servo-core-0.12.21.jar:/Users/Mike/.m2/repository/com/google/guava/guava/19.0/guava-19.0.jar:/Users/Mike/.m2/repository/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar:/Users/Mike/.m2/repository/org/apache/httpcomponents/httpcore/4.4.14/httpcore-4.4.14.jar:/Users/Mike/.m2/repository/commons-codec/commons-codec/1.15/commons-codec-1.15.jar:/Users/Mike/.m2/repository/commons-configuration/commons-configuration/1.10/commons-configuration-1.10.jar:/Users/Mike/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/Users/Mike/.m2/repository/com/google/inject/guice/4.1.0/guice-4.1.0.jar:/Users/Mike/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/Users/Mike/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/Mike/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.11.4/jackson-annotations-2.11.4.jar:/Users/Mike/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.11.4/jackson-core-2.11.4.jar:/Users/Mike/.m2/repository/org/codehaus/jettison/jettison/1.4.0/jettison-1.4.0.jar:/Users/Mike/.m2/repository/com/netflix/eureka/eureka-core/1.10.10/eureka-core-1.10.10.jar:/Users/Mike/.m2/repository/com/fasterxml/woodstox/woodstox-core/5.3.0/woodstox-core-5.3.0.jar:/Users/Mike/.m2/repository/org/codehaus/woodstox/stax2-api/4.2/stax2-api-4.2.jar:/Users/Mike/.m2/repository/org/springframework/cloud/spring-cloud-starter-loadbalancer/3.0.0/spring-cloud-starter-loadbalancer-3.0.0.jar:/Users/Mike/.m2/repository/org/springframework/cloud/spring-cloud-loadbalancer/3.0.0/spring-cloud-loadbalancer-3.0.0.jar:/Users/Mike/.m2/repository/org/springframework/boot/spring-boot-starter-validation/2.4.2/spring-boot-starter-validation-2.4.2.jar:/Users/Mike/.m2/repository/org/hibernate/validator/hibernate-validator/6.1.7.Final/hibernate-validator-6.1.7.Final.jar:/Users/Mike/.m2/repository/jakarta/validation/jakarta.validation-api/2.0.2/jakarta.validation-api-2.0.2.jar:/Users/Mike/.m2/repository/org/jboss/logging/jboss-logging/3.4.1.Final/jboss-logging-3.4.1.Final.jar:/Users/Mike/.m2/repository/com/fasterxml/classmate/1.5.1/classmate-1.5.1.jar:/Users/Mike/.m2/repository/io/projectreactor/reactor-core/3.4.2/reactor-core-3.4.2.jar:/Users/Mike/.m2/repository/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar:/Users/Mike/.m2/repository/io/projectreactor/addons/reactor-extra/3.4.2/reactor-extra-3.4.2.jar:/Users/Mike/.m2/repository/org/springframework/boot/spring-boot-starter-cache/2.4.2/spring-boot-starter-cache-2.4.2.jar:/Users/Mike/.m2/repository/org/springframework/spring-context-support/5.3.3/spring-context-support-5.3.3.jar:/Users/Mike/.m2/repository/com/stoyanr/evictor/1.0.0/evictor-1.0.0.jar:/Users/Mike/.m2/repository/com/thoughtworks/xstream/xstream/1.4.14/xstream-1.4.14.jar:/Users/Mike/.m2/repository/xmlpull/xmlpull/1.1.3.1/xmlpull-1.1.3.1.jar:/Users/Mike/.m2/repository/xpp3/xpp3_min/1.1.4c/xpp3_min-1.1.4c.jar:/Users/Mike/.m2/repository/org/springframework/cloud/spring-cloud-starter-openfeign/3.0.0/spring-cloud-starter-openfeign-3.0.0.jar:/Users/Mike/.m2/repository/org/springframework/cloud/spring-cloud-openfeign-core/3.0.0/spring-cloud-openfeign-core-3.0.0.jar:/Users/Mike/.m2/repository/org/springframework/boot/spring-boot-starter-aop/2.4.2/spring-boot-starter-aop-2.4.2.jar:/Users/Mike/.m2/repository/org/aspectj/aspectjweaver/1.9.6/aspectjweaver-1.9.6.jar:/Users/Mike/.m2/repository/io/github/openfeign/form/feign-form-spring/3.8.0/feign-form-spring-3.8.0.jar:/Users/Mike/.m2/repository/io/github/openfeign/form/feign-form/3.8.0/feign-form-3.8.0.jar:/Users/Mike/.m2/repository/commons-fileupload/commons-fileupload/1.4/commons-fileupload-1.4.jar:/Users/Mike/.m2/repository/commons-io/commons-io/2.2/commons-io-2.2.jar:/Users/Mike/.m2/repository/org/springframework/cloud/spring-cloud-commons/3.0.0/spring-cloud-commons-3.0.0.jar:/Users/Mike/.m2/repository/org/springframework/security/spring-security-crypto/5.4.2/spring-security-crypto-5.4.2.jar:/Users/Mike/.m2/repository/io/github/openfeign/feign-core/10.10.1/feign-core-10.10.1.jar:/Users/Mike/.m2/repository/io/github/openfeign/feign-slf4j/10.10.1/feign-slf4j-10.10.1.jar:/Users/Mike/.m2/repository/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.jar:/Users/Mike/.m2/repository/org/springframework/spring-core/5.3.3/spring-core-5.3.3.jar:/Users/Mike/.m2/repository/org/springframework/spring-jcl/5.3.3/spring-jcl-5.3.3.jar:/Users/Mike/.m2/repository/org/projectlombok/lombok/1.18.16/lombok-1.18.16.jar:/Users/Mike/.m2/repository/com/google/code/gson/gson/2.8.6/gson-2.8.6.jar:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar com.enersave.ack.AckApplication
Connected to the target VM, address: '127.0.0.1:53072', transport: 'socket'

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.4.2)

2021-01-16 17:41:20.420  INFO 15255 --- [           main] com.enersave.ack.AckApplication          : Starting AckApplication using Java 1.8.0_181 on MikesdeMacBook-Pro.local with PID 15255 (/Users/Mike/Documents/OnlineCourse/SCI/Projects/EnerSave/EnerSave/federated_learning_enersave/EnerSave-Cloud/ack-service/target/classes started by Mike in /Users/Mike/Documents/OnlineCourse/SCI/Projects/EnerSave/EnerSave/federated_learning_enersave/EnerSave-Cloud)
2021-01-16 17:41:20.425  INFO 15255 --- [           main] com.enersave.ack.AckApplication          : No active profile set, falling back to default profiles: default
2021-01-16 17:41:22.213  INFO 15255 --- [           main] o.s.cloud.context.scope.GenericScope     : BeanFactory id=0e1b3eaf-af1c-391a-982b-c1fead9956e4
2021-01-16 17:41:22.269  INFO 15255 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'com.enersave.ack.client.NotificationClient' of type [org.springframework.cloud.openfeign.FeignClientFactoryBean] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2021-01-16 17:41:22.658  INFO 15255 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 65315 (http)
2021-01-16 17:41:22.672  INFO 15255 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-01-16 17:41:22.672  INFO 15255 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.41]
2021-01-16 17:41:22.850  INFO 15255 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2021-01-16 17:41:22.850  INFO 15255 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 2290 ms
2021-01-16 17:41:23.310  INFO 15255 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2021-01-16 17:41:23.699  INFO 15255 --- [           main] DiscoveryClientOptionalArgsConfiguration : Eureka HTTP Client uses RestTemplate.
2021-01-16 17:41:23.778  WARN 15255 --- [           main] iguration$LoadBalancerCaffeineWarnLogger : Spring Cloud LoadBalancer is currently working with the default cache. You can switch to using Caffeine cache, by adding it to the classpath.
2021-01-16 17:41:23.862  INFO 15255 --- [           main] o.s.c.n.eureka.InstanceInfoFactory       : Setting initial instance status as: STARTING
2021-01-16 17:41:23.913  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Initializing Eureka in region us-east-1
2021-01-16 17:41:23.922  INFO 15255 --- [           main] c.n.d.s.r.aws.ConfigClusterResolver      : Resolving eureka endpoints via configuration
2021-01-16 17:41:23.952  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Disable delta property : false
2021-01-16 17:41:23.952  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Single vip registry refresh property : null
2021-01-16 17:41:23.952  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Force full registry fetch : false
2021-01-16 17:41:23.952  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Application is null : false
2021-01-16 17:41:23.952  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Registered Applications size is zero : true
2021-01-16 17:41:23.952  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Application version is -1: true
2021-01-16 17:41:23.952  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Getting all instance registry info from the eureka server
2021-01-16 17:41:24.305  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : The response status is 200
2021-01-16 17:41:24.308  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Starting heartbeat executor: renew interval is: 30
2021-01-16 17:41:24.313  INFO 15255 --- [           main] c.n.discovery.InstanceInfoReplicator     : InstanceInfoReplicator onDemand update allowed rate per min is 4
2021-01-16 17:41:24.321  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Discovery Client initialized at timestamp 1610790084319 with initial instances count: 0
2021-01-16 17:41:24.322  INFO 15255 --- [           main] o.s.c.n.e.s.EurekaServiceRegistry        : Registering application ACK-SERVICE with eureka with status UP
2021-01-16 17:41:24.323  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Saw local status change event StatusChangeEvent [timestamp=1610790084323, current=UP, previous=STARTING]
2021-01-16 17:41:24.326  INFO 15255 --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ACK-SERVICE/192.168.0.102:ack-service:65315: registering service...
2021-01-16 17:41:24.369  INFO 15255 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 65315 (http) with context path ''
2021-01-16 17:41:24.371  INFO 15255 --- [           main] .s.c.n.e.s.EurekaAutoServiceRegistration : Updating port to 65315
2021-01-16 17:41:24.374  INFO 15255 --- [           main] com.enersave.ack.handler.CoreAckHandler  : ******************** Application prepared
2021-01-16 17:41:24.385  INFO 15255 --- [           main] com.enersave.ack.AckApplication          : Started AckApplication in 5.041 seconds (JVM running for 6.708)
2021-01-16 17:41:24.387  INFO 15255 --- [           main] com.enersave.ack.handler.CoreAckHandler  : ******************** On application started
2021-01-16 17:41:24.392  INFO 15255 --- [           main] com.enersave.ack.handler.CoreAckHandler  : ******************** On application ready
2021-01-16 17:41:24.395  INFO 15255 --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ACK-SERVICE/192.168.0.102:ack-service:65315 - registration status: 204
2021-01-16 17:41:24.684  WARN 15255 --- [oundedElastic-1] o.s.c.l.core.RoundRobinLoadBalancer      : No servers available for service: notification-service
2021-01-16 17:41:24.685  WARN 15255 --- [           main] .s.c.o.l.FeignBlockingLoadBalancerClient : Load balancer does not contain an instance for the service notification-service
2021-01-16 17:41:24.740 ERROR 15255 --- [           main] o.s.boot.SpringApplication               : Application run failed

feign.FeignException$ServiceUnavailable: [503] during [POST] to [http://notification-service/add_client_info] [NotificationClient#addClientInfo(IntraNotificationClientInfo)]: [Load balancer does not contain an instance for the service notification-service]
    at feign.FeignException.serverErrorStatus(FeignException.java:237) ~[feign-core-10.10.1.jar:na]
    at feign.FeignException.errorStatus(FeignException.java:180) ~[feign-core-10.10.1.jar:na]
    at feign.FeignException.errorStatus(FeignException.java:169) ~[feign-core-10.10.1.jar:na]
    at feign.codec.ErrorDecoder$Default.decode(ErrorDecoder.java:92) ~[feign-core-10.10.1.jar:na]
    at feign.AsyncResponseHandler.handleResponse(AsyncResponseHandler.java:96) ~[feign-core-10.10.1.jar:na]
    at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:138) ~[feign-core-10.10.1.jar:na]
    at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:89) ~[feign-core-10.10.1.jar:na]
    at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:100) ~[feign-core-10.10.1.jar:na]
    at com.sun.proxy.$Proxy79.addClientInfo(Unknown Source) ~[na:na]
    at com.enersave.ack.handler.CoreAckHandler.onApplicationReady(CoreAckHandler.java:55) ~[classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_181]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_181]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_181]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_181]
    at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:312) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:197) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:160) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:426) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:383) ~[spring-context-5.3.3.jar:5.3.3]
    at org.springframework.boot.context.event.EventPublishingRunListener.running(EventPublishingRunListener.java:111) ~[spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplicationRunListeners.lambda$running$6(SpringApplicationRunListeners.java:79) ~[spring-boot-2.4.2.jar:2.4.2]
    at java.util.ArrayList.forEach(ArrayList.java:1257) ~[na:1.8.0_181]
    at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:117) ~[spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:111) ~[spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplicationRunListeners.running(SpringApplicationRunListeners.java:79) ~[spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:341) [spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1311) [spring-boot-2.4.2.jar:2.4.2]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1300) [spring-boot-2.4.2.jar:2.4.2]
    at com.enersave.ack.AckApplication.main(AckApplication.java:14) [classes/:na]

2021-01-16 17:41:24.746  INFO 15255 --- [           main] o.s.c.n.e.s.EurekaServiceRegistry        : Unregistering application ACK-SERVICE with eureka with status DOWN
2021-01-16 17:41:24.746  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Saw local status change event StatusChangeEvent [timestamp=1610790084746, current=DOWN, previous=UP]
2021-01-16 17:41:24.747  INFO 15255 --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ACK-SERVICE/192.168.0.102:ack-service:65315: registering service...
2021-01-16 17:41:24.758  INFO 15255 --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ACK-SERVICE/192.168.0.102:ack-service:65315 - registration status: 204
2021-01-16 17:41:26.040  INFO 15255 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'applicationTaskExecutor'
2021-01-16 17:41:26.045  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Shutting down DiscoveryClient ...
2021-01-16 17:41:29.053  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Unregistering ...
2021-01-16 17:41:29.063  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_ACK-SERVICE/192.168.0.102:ack-service:65315 - deregister  status: 200
2021-01-16 17:41:29.069  INFO 15255 --- [           main] com.netflix.discovery.DiscoveryClient    : Completed shut down of DiscoveryClient
OlgaMaciaszek commented 3 years ago

We rely on information from a service registry (i.e., Eureka). Until it provides us with instances for the serviceId in question, we will not be able to pick an instance while load-balancing. One thing that could help you is to reduce the caching time (https://docs.spring.io/spring-cloud-commons/docs/current/reference/html/#loadbalancer-cache-configuration) so that the map of the available instance will be recreated more often based on new calls to ServiceRegistry.

mikemikezhu commented 3 years ago

@OlgaMaciaszek Thank you so much for your useful suggestions! Since in our application, we registered the service into the service registry (e.g. Eureka) immediately after Spring application is ready (e.g. @EventListener(ApplicationReadyEvent.class)), I discovered if slowly starting each service one by one, then the issue is gone. Therefore, it might be caused by the serviceId is not ready in the service registry, which makes it unable to pick the instance while load-balancing. However, I would like to ask whether Spring Cloud has any callback or notification which could notifies other services that load-balancing is ready and it is safe to pick an instance with serviceId from the service registry? If not, is it beneficial to have such feature in the later versions? Thank you so much again and look forward to your reply!

OlgaMaciaszek commented 3 years ago

The problem here seems to be with the services not being ready in the ServiceRegistry, which is a whole different architecture component and often comes from an entirely different project/ library that we do not maintain (such as Eureka). The best idea might be to check if your services are ready in the service registry of choice (in this case, Eureka). If the healthy instances are available there, you can run your requests. From the solutions that we provide, you might also try using LoadBalancer Retries that will allow you to retry the call several times if there are any issues.

spring-cloud-issues commented 3 years ago

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

spring-cloud-issues commented 3 years ago

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

Girish8262 commented 11 months ago

check service linked

OlgaMaciaszek commented 11 months ago

@Girish8262 which link? Please repost.

spring-cloud-issues commented 11 months ago

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

spring-cloud-issues commented 11 months ago

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.