apache / servicecomb-service-center

A standalone service center to allow services to register their instance information and to discover providers of a given service
Apache License 2.0
1.34k stars 343 forks source link

掉电重启后ServiceCenter实例注册失败 #1146

Open qinyushun opened 3 years ago

qinyushun commented 3 years ago

Describe the bug 虚机掉电重启后ServiceCenter实例获取为空,ServiceCenter自注册失败

To Reproduce Steps to reproduce the behavior: 1、下电虚拟机并重新启动 2、应用连接ServiceCenter并调用接口获取ServiceCenter实例信息 3、获取的ServiceCenter实例信息为空 4、重启ServiceCenter(重启前,步骤3结果一直为空) 5、可以正常获取ServiceCenter实例

Platform And Runtime (please complete the following information): Platform OS/Arch: linux/amd64 Browser [e.g. chrome, safari] Version 0.0 Runtime

Version service-center version: service-center 1.3.0 Go version: go1.16.5 Additional context Add any other context about the problem here.

Logs ServiceCenter日志如下: 2021-08-16T08:12:38.117+0800 WARN clientv3/logger.go:92 Failed to dial 192.168.10.10:2379: grpc: the connection is closing; please retry. 2021-08-16T08:12:38.142+0800 ERROR etcd/listwatch_inner.go:75 watch prefix /cse-sr/inst/leases/ failed, start rev: 13+1->15->0 {"error": "etcdserver: mvcc: required revision has been compacted"} 2021-08-16T08:12:38.142+0800 ERROR etcd/listwatch_inner.go:75 watch prefix /cse-sr/inst/files/ failed, start rev: 13+1->15->0 {"error": "etcdserver: mvcc: required revision has been compacted"} 2021-08-16T08:12:38.142+0800 ERROR etcd/listwatch_inner.go:75 watch prefix /cse-sr/ms/schema-sum/ failed, start rev: 13+1->13->0 {"error": "etcdserver: mvcc: required revision has been compacted"} 2021-08-16T08:12:38.594+0800 INFO event/service_event_handler.go:64 caught [CREATE] service[20344d4ef9c011eb82de286ed49eac32][/sdn/PCE/0.1.11] event 2021-08-16T08:12:38.696+0800 INFO event/service_event_handler.go:64 caught [DELETE] service[9df0b8befe2611eb8320286ed49ead76][production/default/SERVICECENTER/1.3.0] event 2021-08-16T08:13:00.899+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:13:00.899+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered 2021-08-16T08:13:31.008+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:13:31.008+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered 2021-08-16T08:13:31.032+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist 2021-08-16T08:13:31.032+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed 2021-08-16T08:13:31.032+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."} Connection closing...Socket close.ROR service/instance.go:308 heartbeat failed, instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa]. operator {"error": "Instance does not exist(leaseId not exist, instance not exist.)"} Connection closed by foreign host.ROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:14:01.061+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered Disconnected from remote host(8.35.61.40:22) at 22:04:27.ice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist 2021-08-16T08:14:01.066+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed Type `help' to learn how to use Xshell prompt.d/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."} [C:\~]$ ce not exist.)"} 2021-08-16T08:14:31.092+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:14:31.093+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered 2021-08-16T08:14:31.104+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist 2021-08-16T08:14:31.104+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed 2021-08-16T08:14:31.104+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."} 2021-08-16T08:14:46.517+0800 INFO event/instance_event_handler.go:79 caught [DELETE] service[58383baaf9be11eba0e5286ed49eabaa][production/default/SERVICECENTER/1.3.0] instance[47c8305efd5d11eb878e286ed49eabaa] event, endpoints [rest://192.168.10.10:30100/?sslEnabled=true] 2021-08-16T08:14:46.517+0800 INFO event/instance_event_handler.go:79 caught [DELETE] service[58383baaf9be11eba0e5286ed49eabaa][production/default/SERVICECENTER/1.3.0] instance[4740529afd5d11eba5dd286ed49ead76] event, endpoints [rest://192.168.10.8:30100/?sslEnabled=true] 2021-08-16T08:14:46.517+0800 INFO event/instance_event_handler.go:79 caught [DELETE] service[58383baaf9be11eba0e5286ed49eabaa][production/default/SERVICECENTER/1.3.0] instance[498abd77fd5d11eb8bc3286ed49eac32] event, endpoints [rest://192.168.10.9:30100/?sslEnabled=true] 2021-08-16T08:15:01.229+0800 ERROR service/instance.go:308 heartbeat failed, instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa]. operator {"error": "Instance does not exist(leaseId not exist, instance not exist.)"} 2021-08-16T08:15:01.229+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:15:01.229+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered 2021-08-16T08:15:01.260+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist 2021-08-16T08:15:01.260+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed 2021-08-16T08:15:01.261+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."} 2021-08-16T08:15:31.279+0800 ERROR service/instance.go:308 heartbeat failed, instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa]. operator {"error": "Instance does not exist(leaseId not exist, instance not exist.)"} 2021-08-16T08:15:31.279+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:15:31.279+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered 2021-08-16T08:15:31.287+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist 2021-08-16T08:15:31.287+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed 2021-08-16T08:15:31.288+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."}

diwenzheng commented 3 years ago

这个很明显是你的etcd与注册中心断开了

tianxiaoliang commented 3 years ago

这个很明显是你的etcd与注册中心断开了

感谢热心的回复,深入参与社区,可以加下我的微信tianxiaoliang2017