weibocom / motan

A cross-language remote procedure call(RPC) framework for rapid development of high performance distributed services.
Other
5.89k stars 1.78k forks source link

流量切换异常 #1026

Closed zxn-git closed 1 year ago

zxn-git commented 1 year ago

管理后台: image image

zk节点:

image

问题:

  1. 如果不配置流量切换,启动没问题也可以连接上server
  2. 在后台配置好流量切换后,在启动client就会失败
  3. 在后台删除流量切换,依然会启动失败,只能收到删除zk的command节点

启动客户端异常: 19:29:15.876 [main] WARN warn - [ZookeeperRegistry] false to subscribe motan://10.3.0.32:0/com.mtan.interf.FooService?group=motan-ulive-rpc1 from zookeeper://127.0.0.1:2181/com.weibo.api.motan.registry.RegistryService?group=default_rpc com.weibo.api.motan.exception.MotanFrameworkException: error_message: Failed to discover command motan://10.3.0.32:0/com.mtan.interf.FooService?group=motan-ulive-rpc1 from zookeeper(zookeeper://127.0.0.1:2181/com.weibo.api.motan.registry.RegistryService?group=default_rpc), cause: java.io.StreamCorruptedException: invalid stream header: 7B22636C, status: 503, error_code: 20001,r=null at com.weibo.api.motan.registry.zookeeper.ZookeeperRegistry.discoverCommand(ZookeeperRegistry.java:214) at com.weibo.api.motan.registry.support.command.CommandFailbackRegistry.doDiscover(CommandFailbackRegistry.java:70) at com.weibo.api.motan.registry.support.command.CommandFailbackRegistry.doSubscribe(CommandFailbackRegistry.java:49) at com.weibo.api.motan.registry.support.AbstractRegistry.subscribe(AbstractRegistry.java:105) at com.weibo.api.motan.registry.support.FailbackRegistry.subscribe(FailbackRegistry.java:119) at com.weibo.api.motan.cluster.support.ClusterSupport.init(ClusterSupport.java:108) at com.weibo.api.motan.config.handler.SimpleConfigHandler.buildClusterSupport(SimpleConfigHandler.java:56) at com.weibo.api.motan.config.RefererConfig.createClusterSupport(RefererConfig.java:196) at com.weibo.api.motan.config.RefererConfig.initRef(RefererConfig.java:138) at com.weibo.api.motan.config.RefererConfig.getRef(RefererConfig.java:94) at com.weibo.api.motan.config.springsupport.RefererConfigBean.getObject(RefererConfigBean.java:40) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1646) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:254) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1054) at com.mtan.ThreadPoolFullClient.main(ThreadPoolFullClient.java:14) Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'remoteService': FactoryBean threw exception on object creation; nested exception is com.weibo.api.motan.exception.MotanFrameworkException: error_message: [ZookeeperRegistry] false to subscribe motan://10.3.0.32:0/com.mtan.interf.FooService?group=motan-ulive-rpc1 from zookeeper://127.0.0.1:2181/com.weibo.api.motan.registry.RegistryService?group=default_rpc, status: 503, error_code: 20001,r=null at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1646) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:254) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1054) at com.mtan.ThreadPoolFullClient.main(ThreadPoolFullClient.java:14) Caused by: com.weibo.api.motan.exception.MotanFrameworkException: error_message: [ZookeeperRegistry] false to subscribe motan://10.3.0.32:0/com.mtan.interf.FooService?group=motan-ulive-rpc1 from zookeeper://127.0.0.1:2181/com.weibo.api.motan.registry.RegistryService?group=default_rpc, status: 503, error_code: 20001,r=null at com.weibo.api.motan.registry.support.FailbackRegistry.subscribe(FailbackRegistry.java:126) at com.weibo.api.motan.cluster.support.ClusterSupport.init(ClusterSupport.java:108) at com.weibo.api.motan.config.handler.SimpleConfigHandler.buildClusterSupport(SimpleConfigHandler.java:56) at com.weibo.api.motan.config.RefererConfig.createClusterSupport(RefererConfig.java:196) at com.weibo.api.motan.config.RefererConfig.initRef(RefererConfig.java:138) at com.weibo.api.motan.config.RefererConfig.getRef(RefererConfig.java:94) at com.weibo.api.motan.config.springsupport.RefererConfigBean.getObject(RefererConfigBean.java:40) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178) ... 6 more Caused by: com.weibo.api.motan.exception.MotanFrameworkException: error_message: Failed to discover command motan://10.3.0.32:0/com.mtan.interf.FooService?group=motan-ulive-rpc1 from zookeeper(zookeeper://127.0.0.1:2181/com.weibo.api.motan.registry.RegistryService?group=default_rpc), cause: java.io.StreamCorruptedException: invalid stream header: 7B22636C, status: 503, error_code: 20001,r=null at com.weibo.api.motan.registry.zookeeper.ZookeeperRegistry.discoverCommand(ZookeeperRegistry.java:214) at com.weibo.api.motan.registry.support.command.CommandFailbackRegistry.doDiscover(CommandFailbackRegistry.java:70) at com.weibo.api.motan.registry.support.command.CommandFailbackRegistry.doSubscribe(CommandFailbackRegistry.java:49) at com.weibo.api.motan.registry.support.AbstractRegistry.subscribe(AbstractRegistry.java:105) at com.weibo.api.motan.registry.support.FailbackRegistry.subscribe(FailbackRegistry.java:119) ... 13 more

zxn-git commented 1 year ago

client、server motan版本:

1.1.14

管理后台: 拉的最新github代码

zxn-git commented 1 year ago

服务启动后,在控制台配置:client报错 20:52:27.642 [ZkClient-EventThread-13-127.0.0.1:2181] ERROR org.I0Itec.zkclient.ZkEventThread - Error handling event ZkEvent[Data of /motan/motan-ulive-rpc1/command changed sent to com.weibo.api.motan.registry.zookeeper.ZookeeperRegistry$3@20b8330a] org.I0Itec.zkclient.exception.ZkMarshallingError: java.io.StreamCorruptedException: invalid stream header: 7B22636C at org.I0Itec.zkclient.serialize.SerializableSerializer.deserialize(SerializableSerializer.java:37) at org.I0Itec.zkclient.ZkClient.derializable(ZkClient.java:740) at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:773) at org.I0Itec.zkclient.ZkClient$6.run(ZkClient.java:546) at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71) Caused by: java.io.StreamCorruptedException: invalid stream header: 7B22636C at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:857) at java.io.ObjectInputStream.(ObjectInputStream.java:349) at org.I0Itec.zkclient.serialize.TcclAwareObjectIputStream.(TcclAwareObjectIputStream.java:30) at org.I0Itec.zkclient.serialize.SerializableSerializer.deserialize(SerializableSerializer.java:31) ... 4 common frames omitted 20:52:27.642 [ZkClient-EventThread-13-127.0.0.1:2181] DEBUG org.I0Itec.zkclient.ZkEventThread - Delivering event #2 done

rayzhang0603 commented 1 year ago

看异常应该是管理后台和client使用的zkclient的SerializableSerializer不同导致的,后台写入命令使用的编码方式与client使用的解码方式不同,导致无法解析流量指令。

可以从下面几个方面排查一下问题:

  1. 管理后台和client实际使用的zk相关jar包版本是否一致。比如org.apache.zookeepercom.101tec.zkclient是否因依赖冲突使用了不同的版本。
  2. 管理后台和client使用的zkregistry是否一致,即regProtocol的值同为“zookeeper”或者同为“zk”,这是motan提供的两种实现
    • zookeeper:使用ZookeeperRegistryFactorySPI实现类,zkclient使用默认的“SerializableSerializer”编码
    • zk:使用ZookeeperStringSerializerRegistryFactorySPI实现类,zkclient直接使用utf8的string编码。
  3. 如果是其他原因可以参考上面两个SPI扩展实现自己指定的编解码方式,spi扩展方式可以参考这里
zxn-git commented 1 year ago

谢谢,已解决 管理后台是直接下载源码启动的,ZkClient初始化用的是StringSerializer,client默认使用SerializableSerializer