Describe the bug
虚机掉电重启后ServiceCenter实例获取为空,ServiceCenter自注册失败
To Reproduce
Steps to reproduce the behavior:
1、下电虚拟机并重新启动
2、应用连接ServiceCenter并调用接口获取ServiceCenter实例信息
3、获取的ServiceCenter实例信息为空
4、重启ServiceCenter(重启前,步骤3结果一直为空)
5、可以正常获取ServiceCenter实例
Platform And Runtime (please complete the following information):Platform
OS/Arch: linux/amd64
Browser [e.g. chrome, safari]
Version 0.0
Runtime
Version
service-center version: service-center 1.3.0
Go version: go1.16.5
Additional context
Add any other context about the problem here.
LogsServiceCenter日志如下:
2021-08-16T08:12:38.117+0800 WARN clientv3/logger.go:92 Failed to dial 192.168.10.10:2379: grpc: the connection is closing; please retry.
2021-08-16T08:12:38.142+0800 ERROR etcd/listwatch_inner.go:75 watch prefix /cse-sr/inst/leases/ failed, start rev: 13+1->15->0 {"error": "etcdserver: mvcc: required revision has been compacted"}
2021-08-16T08:12:38.142+0800 ERROR etcd/listwatch_inner.go:75 watch prefix /cse-sr/inst/files/ failed, start rev: 13+1->15->0 {"error": "etcdserver: mvcc: required revision has been compacted"}
2021-08-16T08:12:38.142+0800 ERROR etcd/listwatch_inner.go:75 watch prefix /cse-sr/ms/schema-sum/ failed, start rev: 13+1->13->0 {"error": "etcdserver: mvcc: required revision has been compacted"}
2021-08-16T08:12:38.594+0800 INFO event/service_event_handler.go:64 caught [CREATE] service[20344d4ef9c011eb82de286ed49eac32][/sdn/PCE/0.1.11] event
2021-08-16T08:12:38.696+0800 INFO event/service_event_handler.go:64 caught [DELETE] service[9df0b8befe2611eb8320286ed49ead76][production/default/SERVICECENTER/1.3.0] event
2021-08-16T08:13:00.899+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed
2021-08-16T08:13:00.899+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered
2021-08-16T08:13:31.008+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed
2021-08-16T08:13:31.008+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered
2021-08-16T08:13:31.032+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist
2021-08-16T08:13:31.032+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed
2021-08-16T08:13:31.032+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."}
Connection closing...Socket close.ROR service/instance.go:308 heartbeat failed, instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa]. operator {"error": "Instance does not exist(leaseId not exist, instance not exist.)"}
Connection closed by foreign host.ROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed
2021-08-16T08:14:01.061+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered
Disconnected from remote host(8.35.61.40:22) at 22:04:27.ice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist
2021-08-16T08:14:01.066+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed
Type `help' to learn how to use Xshell prompt.d/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."}
[C:\~]$ ce not exist.)"}
2021-08-16T08:14:31.092+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed
2021-08-16T08:14:31.093+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered
2021-08-16T08:14:31.104+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist
2021-08-16T08:14:31.104+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed
2021-08-16T08:14:31.104+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."}
2021-08-16T08:14:46.517+0800 INFO event/instance_event_handler.go:79 caught [DELETE] service[58383baaf9be11eba0e5286ed49eabaa][production/default/SERVICECENTER/1.3.0] instance[47c8305efd5d11eb878e286ed49eabaa] event, endpoints [rest://192.168.10.10:30100/?sslEnabled=true]
2021-08-16T08:14:46.517+0800 INFO event/instance_event_handler.go:79 caught [DELETE] service[58383baaf9be11eba0e5286ed49eabaa][production/default/SERVICECENTER/1.3.0] instance[4740529afd5d11eba5dd286ed49ead76] event, endpoints [rest://192.168.10.8:30100/?sslEnabled=true]
2021-08-16T08:14:46.517+0800 INFO event/instance_event_handler.go:79 caught [DELETE] service[58383baaf9be11eba0e5286ed49eabaa][production/default/SERVICECENTER/1.3.0] instance[498abd77fd5d11eb8bc3286ed49eac32] event, endpoints [rest://192.168.10.9:30100/?sslEnabled=true]
2021-08-16T08:15:01.229+0800 ERROR service/instance.go:308 heartbeat failed, instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa]. operator {"error": "Instance does not exist(leaseId not exist, instance not exist.)"}
2021-08-16T08:15:01.229+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed
2021-08-16T08:15:01.229+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered
2021-08-16T08:15:01.260+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist
2021-08-16T08:15:01.260+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed
2021-08-16T08:15:01.261+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."}
2021-08-16T08:15:31.279+0800 ERROR service/instance.go:308 heartbeat failed, instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa]. operator {"error": "Instance does not exist(leaseId not exist, instance not exist.)"}
2021-08-16T08:15:31.279+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed
2021-08-16T08:15:31.279+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered
2021-08-16T08:15:31.287+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist
2021-08-16T08:15:31.287+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed
2021-08-16T08:15:31.288+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."}
Describe the bug 虚机掉电重启后ServiceCenter实例获取为空,ServiceCenter自注册失败
To Reproduce Steps to reproduce the behavior: 1、下电虚拟机并重新启动 2、应用连接ServiceCenter并调用接口获取ServiceCenter实例信息 3、获取的ServiceCenter实例信息为空 4、重启ServiceCenter(重启前,步骤3结果一直为空) 5、可以正常获取ServiceCenter实例
Platform And Runtime (please complete the following information): Platform OS/Arch: linux/amd64 Browser [e.g. chrome, safari] Version 0.0 Runtime
Version service-center version: service-center 1.3.0 Go version: go1.16.5 Additional context Add any other context about the problem here.
Logs ServiceCenter日志如下: 2021-08-16T08:12:38.117+0800 WARN clientv3/logger.go:92 Failed to dial 192.168.10.10:2379: grpc: the connection is closing; please retry. 2021-08-16T08:12:38.142+0800 ERROR etcd/listwatch_inner.go:75 watch prefix /cse-sr/inst/leases/ failed, start rev: 13+1->15->0 {"error": "etcdserver: mvcc: required revision has been compacted"} 2021-08-16T08:12:38.142+0800 ERROR etcd/listwatch_inner.go:75 watch prefix /cse-sr/inst/files/ failed, start rev: 13+1->15->0 {"error": "etcdserver: mvcc: required revision has been compacted"} 2021-08-16T08:12:38.142+0800 ERROR etcd/listwatch_inner.go:75 watch prefix /cse-sr/ms/schema-sum/ failed, start rev: 13+1->13->0 {"error": "etcdserver: mvcc: required revision has been compacted"} 2021-08-16T08:12:38.594+0800 INFO event/service_event_handler.go:64 caught [CREATE] service[20344d4ef9c011eb82de286ed49eac32][/sdn/PCE/0.1.11] event 2021-08-16T08:12:38.696+0800 INFO event/service_event_handler.go:64 caught [DELETE] service[9df0b8befe2611eb8320286ed49ead76][production/default/SERVICECENTER/1.3.0] event 2021-08-16T08:13:00.899+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:13:00.899+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered 2021-08-16T08:13:31.008+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:13:31.008+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered 2021-08-16T08:13:31.032+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist 2021-08-16T08:13:31.032+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed 2021-08-16T08:13:31.032+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."} Connection closing...Socket close.ROR service/instance.go:308 heartbeat failed, instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa]. operator {"error": "Instance does not exist(leaseId not exist, instance not exist.)"} Connection closed by foreign host.ROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:14:01.061+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered Disconnected from remote host(8.35.61.40:22) at 22:04:27.ice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist 2021-08-16T08:14:01.066+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed Type `help' to learn how to use Xshell prompt.d/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."} [C:\~]$ ce not exist.)"} 2021-08-16T08:14:31.092+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:14:31.093+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered 2021-08-16T08:14:31.104+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist 2021-08-16T08:14:31.104+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed 2021-08-16T08:14:31.104+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."} 2021-08-16T08:14:46.517+0800 INFO event/instance_event_handler.go:79 caught [DELETE] service[58383baaf9be11eba0e5286ed49eabaa][production/default/SERVICECENTER/1.3.0] instance[47c8305efd5d11eb878e286ed49eabaa] event, endpoints [rest://192.168.10.10:30100/?sslEnabled=true] 2021-08-16T08:14:46.517+0800 INFO event/instance_event_handler.go:79 caught [DELETE] service[58383baaf9be11eba0e5286ed49eabaa][production/default/SERVICECENTER/1.3.0] instance[4740529afd5d11eba5dd286ed49ead76] event, endpoints [rest://192.168.10.8:30100/?sslEnabled=true] 2021-08-16T08:14:46.517+0800 INFO event/instance_event_handler.go:79 caught [DELETE] service[58383baaf9be11eba0e5286ed49eabaa][production/default/SERVICECENTER/1.3.0] instance[498abd77fd5d11eb8bc3286ed49eac32] event, endpoints [rest://192.168.10.9:30100/?sslEnabled=true] 2021-08-16T08:15:01.229+0800 ERROR service/instance.go:308 heartbeat failed, instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa]. operator {"error": "Instance does not exist(leaseId not exist, instance not exist.)"} 2021-08-16T08:15:01.229+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:15:01.229+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered 2021-08-16T08:15:01.260+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist 2021-08-16T08:15:01.260+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed 2021-08-16T08:15:01.261+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."} 2021-08-16T08:15:31.279+0800 ERROR service/instance.go:308 heartbeat failed, instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa]. operator {"error": "Instance does not exist(leaseId not exist, instance not exist.)"} 2021-08-16T08:15:31.279+0800 ERROR backend/registry.go:220 update service center instance[9df0b8befe2611eb8320286ed49ead76/a5a5fba8fe2611eb8311286ed49eabaa] heartbeat failed 2021-08-16T08:15:31.279+0800 WARN backend/registry.go:156 service center service[9df0b8befe2611eb8320286ed49ead76] already registered 2021-08-16T08:15:31.287+0800 ERROR service/microservice.go:492 get micro-service[9df0b8befe2611eb8320286ed49ead76] failed, service does not exist 2021-08-16T08:15:31.287+0800 ERROR backend/registry.go:159 query service center service[9df0b8befe2611eb8320286ed49ead76] info failed 2021-08-16T08:15:31.288+0800 ERROR backend/registry.go:226 retry to register[production/default/SERVICECENTER/1.3.0] failed {"error": "service center service file lost."}