apache / dubbo

The java implementation of Apache Dubbo. An RPC and microservice framework.
https://dubbo.apache.org/
Apache License 2.0
40.44k stars 26.42k forks source link

dubbo 2.7.4.1 Failed to receive INITIALIZED event from zookeeper, pls. check if url #5678

Closed shifefiei closed 2 years ago

shifefiei commented 4 years ago

[INFO ] [10:53:36] org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting [DEBUG] [10:53:37] org.apache.curator.CuratorZookeeperClient - Starting [DEBUG] [10:53:37] org.apache.curator.ConnectionState - Starting [DEBUG] [10:53:37] org.apache.curator.ConnectionState - reset [INFO ] [10:53:37] org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED [INFO ] [10:53:37] org.apache.dubbo.remoting.zookeeper.curator.CuratorZookeeperClient - [DUBBO] Curator zookeeper client instance initiated successfully, session id is 2693fa771f35631, dubbo version: 2.7.4.1, current host: 10.1.4.79 [INFO ] [10:53:37] org.apache.dubbo.remoting.zookeeper.ZookeeperTransporter - [DUBBO] No valid zookeeper client found from cache, therefore create a new client for url. zookeeper://10.24.22.83:2181/ConfigCenterConfig?backup=10.26.12.141:2181,10.25.191.213:2181&check=true&config-file=dubbo.properties&group=dubbo&highest-priority=false&namespace=dubbo&timeout=3000, dubbo version: 2.7.4.1, current host: 10.1.4.79 [DEBUG] [10:53:42] org.apache.curator.framework.imps.CuratorFrameworkImpl - Closing [DEBUG] [10:53:42] org.apache.curator.CuratorZookeeperClient - Closing [DEBUG] [10:53:42] org.apache.curator.ConnectionState - Closing [DEBUG] [10:53:42] org.apache.curator.framework.imps.CuratorFrameworkImpl - Closing [DEBUG] [10:53:42] org.apache.curator.framework.imps.CuratorFrameworkImpl - Closing [DEBUG] [10:53:42] org.apache.curator.framework.imps.CuratorFrameworkImpl - Closing [DEBUG] [10:53:42] org.apache.curator.framework.imps.CuratorFrameworkImpl - Closing [DEBUG] [10:53:42] org.apache.curator.framework.imps.CuratorFrameworkImpl - Closing [DEBUG] [10:53:42] org.apache.curator.framework.imps.CuratorFrameworkImpl - Closing [ERROR] [10:53:42] org.springframework.web.context.ContextLoader - Context initialization failed org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'smsSendController': Injection of resource dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'smsServiceImpl': Injection of resource dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'smsRpcService': FactoryBean threw exception on object creation; nested exception is java.lang.IllegalStateException: Failed to receive INITIALIZED event from zookeeper, pls. check if url zookeeper://10.24.22.83:2181/ConfigCenterConfig?backup=10.26.12.141:2181,10.25.191.213:2181&check=true&config-file=dubbo.properties&group=dubbo&highest-priority=false&namespace=dubbo&timeout=3000 is correct at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.postProcessPropertyValues(CommonAnnotationBeanPostProcessor.java:311) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1219) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:551) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:778) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:843) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:541) at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:444) at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:326) at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:771) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:411) at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:763) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:247) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1238) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:706) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:480) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64) at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:95) at org.eclipse.jetty.server.Server.doStart(Server.java:277) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64) at Runner.main(Runner.java:53) Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'smsServiceImpl': Injection of resource dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'smsRpcService': FactoryBean threw exception on object creation; nested exception is java.lang.IllegalStateException: Failed to receive INITIALIZED event from zookeeper, pls. check if url zookeeper://10.24.22.83:2181/ConfigCenterConfig?backup=10.26.12.141:2181,10.25.191.213:2181&check=true&config-file=dubbo.properties&group=dubbo&highest-priority=false&namespace=dubbo&timeout=3000 is correct at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.postProcessPropertyValues(CommonAnnotationBeanPostProcessor.java:311) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1219) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:551) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1199) at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1123) at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1021) at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.autowireResource(CommonAnnotationBeanPostProcessor.java:508) at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.getResource(CommonAnnotationBeanPostProcessor.java:486) at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor$ResourceElement.getResourceToInject(CommonAnnotationBeanPostProcessor.java:615) at org.springframework.beans.factory.annotation.InjectionMetadata$InjectedElement.inject(InjectionMetadata.java:169) at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88) at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.postProcessPropertyValues(CommonAnnotationBeanPostProcessor.java:308) ... 25 more Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'smsRpcService': FactoryBean threw exception on object creation; nested exception is java.lang.IllegalStateException: Failed to receive INITIALIZED event from zookeeper, pls. check if url zookeeper://10.24.22.83:2181/ConfigCenterConfig?backup=10.26.12.141:2181,10.25.191.213:2181&check=true&config-file=dubbo.properties&group=dubbo&highest-priority=false&namespace=dubbo&timeout=3000 is correct at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1601) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:254) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.autowireResource(CommonAnnotationBeanPostProcessor.java:512) at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.getResource(CommonAnnotationBeanPostProcessor.java:486) at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor$ResourceElement.getResourceToInject(CommonAnnotationBeanPostProcessor.java:615) at org.springframework.beans.factory.annotation.InjectionMetadata$InjectedElement.inject(InjectionMetadata.java:169) at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88) at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.postProcessPropertyValues(CommonAnnotationBeanPostProcessor.java:308) ... 41 more Caused by: java.lang.IllegalStateException: Failed to receive INITIALIZED event from zookeeper, pls. check if url zookeeper://10.24.22.83:2181/ConfigCenterConfig?backup=10.26.12.141:2181,10.25.191.213:2181&check=true&config-file=dubbo.properties&group=dubbo&highest-priority=false&namespace=dubbo&timeout=3000 is correct at org.apache.dubbo.configcenter.support.zookeeper.ZookeeperDynamicConfiguration.(ZookeeperDynamicConfiguration.java:69) at org.apache.dubbo.configcenter.support.zookeeper.ZookeeperDynamicConfigurationFactory.createDynamicConfiguration(ZookeeperDynamicConfigurationFactory.java:37) at org.apache.dubbo.configcenter.AbstractDynamicConfigurationFactory.getDynamicConfiguration(AbstractDynamicConfigurationFactory.java:33) at org.apache.dubbo.config.AbstractInterfaceConfig.getDynamicConfiguration(AbstractInterfaceConfig.java:315) at org.apache.dubbo.config.AbstractInterfaceConfig.prepareEnvironment(AbstractInterfaceConfig.java:290) at org.apache.dubbo.config.AbstractInterfaceConfig.startConfigCenter(AbstractInterfaceConfig.java:280) at org.apache.dubbo.config.AbstractInterfaceConfig.lambda$null$7(AbstractInterfaceConfig.java:636) at java.util.Optional.orElseGet(Optional.java:267) at org.apache.dubbo.config.AbstractInterfaceConfig.lambda$useRegistryForConfigIfNecessary$8(AbstractInterfaceConfig.java:620) at java.util.Optional.ifPresent(Optional.java:159) at org.apache.dubbo.config.AbstractInterfaceConfig.useRegistryForConfigIfNecessary(AbstractInterfaceConfig.java:618) at org.apache.dubbo.config.AbstractInterfaceConfig.checkRegistry(AbstractInterfaceConfig.java:208) at org.apache.dubbo.config.ReferenceConfig.createProxy(ReferenceConfig.java:378) at org.apache.dubbo.config.ReferenceConfig.init(ReferenceConfig.java:329) at org.apache.dubbo.config.ReferenceConfig.get(ReferenceConfig.java:250) at org.apache.dubbo.config.spring.ReferenceBean.getObject(ReferenceBean.java:73) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) ... 51 more

CodingSinger commented 4 years ago

Please check if your zookeeper is running normally.

shifefiei commented 4 years ago

Please check if your zookeeper is running normally

yes ,it's running normally. Redeployment project can be connect

yipixiaofeiyang commented 4 years ago

you can set System.setProperty(ZKClientConfig.ENABLE_CLIENT_SASL_KEY, "false") before start dubbo service;

tianshuang commented 4 years ago

I also encountered the same problem in dubbo 2.7.2, except that hung was there, guessing that the zk event was lost? #[4314]

tianshuang commented 4 years ago

ZookeeperDynamicConfiguration will be busy wait if zk is unavailable

tianshuang commented 4 years ago

Dubbo 2.7.2 does not set a timeout, when the zk event does not return, the application has been waiting.

linqtosql commented 4 years ago

i have the same problem in dubbo 2.7.7

Kelcin2 commented 4 years ago

Did you resolve this problem?I have also encountered the same problem in dubbo 2.7.8

zondahuman commented 4 years ago

I have also encountered the same problem in dubbo 2.7.8

Kelcin2 commented 4 years ago

I have also encountered the same problem in dubbo 2.7.8

After some of my efforts , I have resolved this problem , just configured this <dubbo:metadata-report /> to disable metadata registration

caojiantong commented 4 years ago

i clone the dubbo-samples-zookeeper, using 2.7.7</dubbo.version> everything is fine,but when i switch it to 2.7.8, it shows "Exception in thread "main" java.lang.IllegalStateException: Failed to receive INITIALIZED event from zookeeper, pls. check if url zookeeper://ip:2181/org.apache.dubbo.metadata.report.MetadataReport?backup=ip1,ip2&client=&dubbo.config-center.root-path=/&timeout=10000 is correct " what can i do to fix the problem

Kelcin2 commented 4 years ago

i clone the dubbo-samples-zookeeper, using 2.7.7</dubbo.version> everything is fine,but when i switch it to 2.7.8, it shows "Exception in thread "main" java.lang.IllegalStateException: Failed to receive INITIALIZED event from zookeeper, pls. check if url zookeeper://ip:2181/org.apache.dubbo.metadata.report.MetadataReport?backup=ip1,ip2&client=&dubbo.config-center.root-path=/&timeout=10000 is correct " what can i do to fix the problem

you can configure like this (modify according to your needs) : <dubbo:metadata-report address="zookeeper://127.0.0.1:2181?init.timeout=60000" timeout="60000"/> or disable metadata registration : <dubbo:metadata-report />

haocao commented 4 years ago

We got the same exception which using 2.7.8 version, and can't resolve it by FlyingHe's suggestion.

you can configure like this (modify according to your needs) :
<dubbo:metadata-report address="zookeeper://127.0.0.1:2181?init.timeout=60000" timeout="60000"/>
or disable metadata registration :
<dubbo:metadata-report />
Kelcin2 commented 4 years ago

We got the same exception which using 2.7.8 version, and can't resolve it by FlyingHe's suggestion.

you can configure like this (modify according to your needs) :
<dubbo:metadata-report address="zookeeper://127.0.0.1:2181?init.timeout=60000" timeout="60000"/>
or disable metadata registration :
<dubbo:metadata-report />

I really resolved this problem By this way,so could you please show your configuration here?

haocao commented 4 years ago

Sure, here is the configuration:

dubbo:
  metadata-report:
    address: zookeeper://127.0.0.1:2181?init.timeout=60000
    timeout: 60000
haocao commented 4 years ago

And the error log is: java.lang.IllegalStateException: Failed to receive INITIALIZED event from zookeeper, pls. check if url zookeeper://localhost:2181/org.apache.dubbo.metadata.report.MetadataReport?dubbo.config-center.root-path=/&init.timeout=60000&timeout=60000 is correct

Kelcin2 commented 4 years ago

And the error log is: java.lang.IllegalStateException: Failed to receive INITIALIZED event from zookeeper, pls. check if url zookeeper://localhost:2181/org.apache.dubbo.metadata.report.MetadataReport?dubbo.config-center.root-path=/&init.timeout=60000&timeout=60000 is correct

Are you sure that your zookeeper is running and it's address is 127.0.0.1:2181 ? if you confirm it is right, maybe you should debug the code for finding out the problem thoroughly .

By the way ,the reason of the problem is that the dubbo need upload some meta data to zookeeper and it wait for time out ,so if you needn't the function you can disable it by configuring empty metadata-report <dubbo:metadata-report />

haocao commented 4 years ago

The zk server address is a fade one, and I'm using the spring boot's yml file to config the metadata-report, do you know how to disable it by the yml file ?

Kelcin2 commented 4 years ago

The zk server address is a fade one, and I'm using the spring boot's yml file to config the metadata-report, do you know how to disable it by the yml file ?

Just like this(Set value to nothing) :

dubbo:
  metadata-report:
    address: 
fungoddd commented 4 years ago

Add configuration: dubbo.metadata-report.address= or dubbo.metadata-report.cycle-report=false

spilledyear commented 4 years ago

@caojiantong 解决了吗? 发现预发跑了几周没问题, 线上有问题

caojiantong commented 4 years ago

@caojiantong 解决了吗? 发现预发跑了几周没问题, 线上有问题 Add configuration: dubbo.metadata-report.cycle-report=false it works for me

spilledyear commented 4 years ago

@caojiantong 你这个是因为 ZK使用默认元数据中心,导致监听了ZK / 节点导致的,看这里 #6780

spilledyear commented 4 years ago

java.lang.IllegalStateException: Failed to receive INITIALIZED event from zookeeper, pls. check if url zookeeper://localhost:2181/org.apache.dubbo.metadata.report.MetadataReport?dubbo.config-center.root-path=/&init.timeout=60000&timeout=60000 is correct

这个错应该是因为 ZK使用默认元数据中心,导致监听了ZK / 节点导致的, #6780

CrazyHZM commented 2 years ago

Try it with the latest version, if you still have problems, you can reopen the issue