search

apache / dubbo

The java implementation of Apache Dubbo. An RPC and microservice framework.
https://dubbo.apache.org/
Apache License 2.0
40.39k stars 26.41k forks source link

The client has been closed and will not reconnect #12625

Open Lingzhou314 opened 1 year ago

Lingzhou314 commented 1 year ago

为什么收到服务变更通知，然后连上了，但是随后又立即关闭了，而且也不再重连了，导致这个消费者永远调不到这个提供者了

这个是启动日志

2023-06-29 18:09:32.786 [nacos.publisher-com.alibaba.nacos.client.naming.event.InstancesChangeEvent] INFO  o.a.d.r.c.e.l.ServiceInstancesChangedListener -[traceId=N/A]ServiceInstancesChangedListener.java:308-  [DUBBO] Received instance no
tification, serviceName: member-service, instances: 1, dubbo version: 3.2.2, current host: 172.22.0.98
2023-06-29 18:09:32.787 [nacos.publisher-com.alibaba.nacos.client.naming.event.InstancesChangeEvent] INFO  o.apache.dubbo.rpc.model.ModuleModel -[traceId=N/A]ModuleModel.java:203-  [DUBBO] Dynamically registering consumer model member-ser
vice/org.apache.dubbo.metadata.MetadataService:1.0.0 into model Dubbo Module[1.1.0], dubbo version: 3.2.2, current host: 172.22.0.98
2023-06-29 18:09:32.802 [NettyClientWorker-5-8] INFO  o.a.d.r.t.netty4.NettyServerHandler -[traceId=N/A]NettyServerHandler.java:75-  [DUBBO] The connection of /172.22.0.102:50051 -> /172.22.0.98:56410 is established., dubbo version: 3.2.2
, current host: 172.22.0.98
2023-06-29 18:09:32.802 [NettyClientWorker-5-8] INFO  o.a.d.r.t.n.NettyConnectionHandler -[traceId=N/A]NettyConnectionHandler.java:100-  [DUBBO] The connection of /172.22.0.98:56410 -> /172.22.0.102:50051 is established., dubbo version: 3
.2.2, current host: 172.22.0.98
2023-06-29 18:09:32.803 [nacos.publisher-com.alibaba.nacos.client.naming.event.InstancesChangeEvent] INFO  o.a.d.r.transport.AbstractClient -[traceId=N/A]AbstractClient.java:228-  [DUBBO] Successfully connect to server /172.22.0.102:50051
 from NettyConnectionClient 172.22.0.98 using dubbo version 3.2.2, channel is NettyChannel [channel=[id: 0xd00daf02, L:/172.22.0.98:56410 - R:/172.22.0.102:50051]], dubbo version: 3.2.2, current host: 172.22.0.98
2023-06-29 18:09:32.803 [nacos.publisher-com.alibaba.nacos.client.naming.event.InstancesChangeEvent] INFO  o.a.d.r.transport.AbstractClient -[traceId=N/A]AbstractClient.java:79-  [DUBBO] Start NettyConnectionClient /172.22.0.98 connect to
 the server /172.22.0.102:50051, dubbo version: 3.2.2, current host: 172.22.0.98
2023-06-29 18:09:33.135 [nacos.publisher-com.alibaba.nacos.client.naming.event.InstancesChangeEvent] INFO  o.a.d.r.t.netty4.NettyChannel -[traceId=N/A]NettyChannel.java:253-  [DUBBO] Close netty channel [id: 0xd00daf02, L:/172.22.0.98:564
10 - R:/172.22.0.102:50051], dubbo version: 3.2.2, current host: 172.22.0.98
2023-06-29 18:09:33.136 [NettyClientWorker-5-8] INFO  o.a.d.r.t.netty4.NettyServerHandler -[traceId=N/A]NettyServerHandler.java:90-  [DUBBO] The connection of /172.22.0.102:50051 -> /172.22.0.98:56410 is disconnected., dubbo version: 3.2.
2, current host: 172.22.0.98
2023-06-29 18:09:33.137 [NettyClientWorker-5-8] INFO  o.a.d.r.t.n.NettyConnectionHandler -[traceId=N/A]NettyConnectionHandler.java:81-  [DUBBO] The client has been closed and will not reconnect. , dubbo version: 3.2.2, current host: 172.2
2.0.98

这个是nacos上的服务提供者变更后的ip 1

songxiaosheng commented 1 year ago

启动时候观察下网络连接状态看下是客户端断开的还是服务端断开的

Lingzhou314 commented 1 year ago

启动时候观察下网络连接状态看下是客户端断开的还是服务端断开的

这个是偶发的，不是必现，具体操作是我用Jenkins同时发布了十几个服务，包括所有消费者和提供者，而且他们都是单实例，但只有这一个消费者有这种错误，其他的消费者都是正常连接的，而且我重启这个消费者也能正常连接。只是不清楚为什么连接上了，又断开了是什么意思，难道是网络抖动的缘故，我们的服务都是部署在k8s里的，是同一个子网

AlbumenJ commented 1 year ago

@EarthChen @icodening @guohao PTAL

icodening commented 1 year ago

@Lingzhou314 贴出日志是一个进程里打出来的吗？从日志中看起来似乎是自己连自己，并且有两个IP？

2023-06-29 18:09:32.802 [NettyClientWorker-5-8] INFO  o.a.d.r.t.netty4.NettyServerHandler -[traceId=N/A]NettyServerHandler.java:75-  [DUBBO] The connection of /172.22.0.102:50051 -> /172.22.0.98:56410 is established., dubbo version: 3.2.2
, current host: 172.22.0.98
2023-06-29 18:09:32.802 [NettyClientWorker-5-8] INFO  o.a.d.r.t.n.NettyConnectionHandler -[traceId=N/A]NettyConnectionHandler.java:100-  [DUBBO] The connection of /172.22.0.98:56410 -> /172.22.0.102:50051 is established., dubbo version: 3
.2.2, current host: 172.22.0.98
Lingzhou314 commented 1 year ago

@Lingzhou314 贴出日志是一个进程里打出来的吗？从日志中看起来似乎是自己连自己，并且有两个IP？

@icodening 是同一个进程里打出来的，98是消费者的ip，102是服务者的ip，这两条日志都是从消费者98这边打印出来的，而且从current host: 172.22.0.98看出这两条日志是同一个ip下的