search

UESTC-KEEP / keep

the project is running at k8s to enhance k8s and kubeedge
4 stars 0 forks source link

edgepublisher长时间工作会停止向edgetwin #4

Open EthanAlban opened 2 years ago

EthanAlban commented 2 years ago

在完全离线的情况下，edgepublisher长时间工作，前期正常，但是一段时间后会出现停止向edgetwin发送数据 考虑两方面问题： ①：sqliter连接是否超时或超次数 ②：edgepublisher逻辑问题

EthanAlban commented 2 years ago

日志记录如下： Content:v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc0001a6180), Cpu:([]cpu.InfoStat)(0xc000291848), CpuUsage:1.1378002528500244, Mem:(mem.VirtualMemoryStat)(0xc0005a65a0), DiskPartitionStat:([]disk.PartitionStat)(0xc0003b1380), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc00026e0b0), NetIOCountersStat:([]net.IOCountersStat)(0xc00000e210), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)}} 2021-12-19 17:24:19 [TRAC] [keep/edge/pkg/edgepublisher/publisher/publisher.go:38 ] send message to edge cloud error: websocket session is nil,try write to edgetwin... edtwin 接收消息 msg:v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc0001a6180), Cpu:([]cpu.InfoStat)(0xc000291848), CpuUsage:1.1378002528500244, Mem:(mem.VirtualMemoryStat)(0xc0005a65a0), DiskPartitionStat:([k.PartitionStat)(0xc0003b1380), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc00026e0b0), NetIOCountersStat:([]net.IOCountersStat)(0xc00000e210), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)} &{0xc0003fdf20} 2021-12-19 17:24:20 [EROR] [keep/edge/pkg/edgepublisher/tunnel/edgetunnel.go:122 ] connect failed: dial tcp 192.168.1.121:20000: connect: no route to host 2021-12-19 17:24:21 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量... edgepublisher 接收消息 msg: "2021-12-19 17:24:21 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..." -------------------------- 发送云端 msg:model.Message{Header:model.MessageHeader{ID:"e4a835b1-8de3-483d-8232-19d831991f82", ParentID:"", Timestamp:1639905861040, ResourceVersion:"", Sync:false}, Router:model.MessageRoute{Source:"lgent", Group:"/log", Operation:"", Resource:""}, Content:"2021-12-19 17:24:21 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..."} 2021-12-19 17:24:21 [TRAC] [keep/edge/pkg/edgepublisher/publisher/publisher.go:38 ] send message to edge cloud error: websocket session is nil,try write to edgetwin... edtwin 接收消息 msg:"2021-12-19 17:24:21 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..." &{0xc0003fdf20} edgepublisher 接收消息 msg: v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc0001a6240), Cpu:([]cpu.InfoStat)(0xc00000e600), CpuUsage:0.8805031446720182, Mem:(mem.VirtualMemoryStat)(0xc00020c120), DiskPartitiot:([]disk.PartitionStat)(0xc000338930), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc000010060), NetIOCountersStat:([]net.IOCountersStat)(0xc000339758), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)} -------------------------- 发送云端 msg:model.Message{Header:model.MessageHeader{ID:"", ParentID:"", Timestamp:0, ResourceVersion:"", Sync:false}, Router:model.MessageRoute{Source:"healthzagent", Group:"", Operation:"", Resource:"Content:v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc0001a6240), Cpu:([]cpu.InfoStat)(0xc00000e600), CpuUsage:0.8805031446720182, Mem:(mem.VirtualMemoryStat)(0xc00020c120), DiskPartitionStat:([]disk.PartitionStat)(0xc000338930), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc000010060), NetIOCountersStat:([]net.IOCountersStat)(0xc000339758), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)}} 2021-12-19 17:24:22 [TRAC] [keep/edge/pkg/edgepublisher/publisher/publisher.go:38 ] send message to edge cloud error: websocket session is nil,try write to edgetwin... edtwin 接收消息 msg:v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc0001a6240), Cpu:([]cpu.InfoStat)(0xc00000e600), CpuUsage:0.8805031446720182, Mem:(mem.VirtualMemoryStat)(0xc00020c120), DiskPartitionStat:([]disk.PartitionStat)(0xc000338930), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc000010060), NetIOCountersStat:([]net.IOCountersStat)(0xc000339758), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)} &{0xc0003fdf20} 2021-12-19 17:24:24 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量... edgepublisher 接收消息 msg: "2021-12-19 17:24:24 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..." -------------------------- 发送云端 msg:model.Message{Header:model.MessageHeader{ID:"26ffa60b-2dfc-41da-8193-3497149a2573", ParentID:"", Timestamp:1639905864049, ResourceVersion:"", Sync:false}, Router:model.MessageRoute{Source:"lgent", Group:"/log", Operation:"", Resource:""}, Content:"2021-12-19 17:24:24 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..."} 2021-12-19 17:24:24 [TRAC] [keep/edge/pkg/edgepublisher/publisher/publisher.go:38 ] send message to edge cloud error: websocket session is nil,try write to edgetwin... edtwin 接收消息 msg:"2021-12-19 17:24:24 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..." &{0xc0003fdf20} edgepublisher 接收消息 msg: v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc0002600c0), Cpu:([]cpu.InfoStat)(0xc000339b60), CpuUsage:2.9262086513991377, Mem:(mem.VirtualMemoryStat)(0xc0002fc240), DiskPartitiot:([]disk.PartitionStat)(0xc00000e540), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc00026e028), NetIOCountersStat:([]net.IOCountersStat)(0xc00000f398), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)} -------------------------- 发送云端 msg:model.Message{Header:model.MessageHeader{ID:"", ParentID:"", Timestamp:0, ResourceVersion:"", Sync:false}, Router:model.MessageRoute{Source:"healthzagent", Group:"", Operation:"", Resource:"Content:v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc0002600c0), Cpu:([]cpu.InfoStat)(0xc000339b60), CpuUsage:2.9262086513991377, Mem:(mem.VirtualMemoryStat)(0xc0002fc240), DiskPartitionStat:([]disk.PartitionStat)(0xc00000e540), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc00026e028), NetIOCountersStat:([]net.IOCountersStat)(0xc00000f398), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)}} 2021-12-19 17:24:25 [TRAC] [keep/edge/pkg/edgepublisher/publisher/publisher.go:38 ] send message to edge cloud error: websocket session is nil,try write to edgetwin... edtwin 接收消息 msg:v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc0002600c0), Cpu:([]cpu.InfoStat)(0xc000339b60), CpuUsage:2.9262086513991377, Mem:(mem.VirtualMemoryStat)(0xc0002fc240), DiskPartitionStat:([k.PartitionStat)(0xc00000e540), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc00026e028), NetIOCountersStat:([]net.IOCountersStat)(0xc00000f398), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)} &{0xc0003fdf20} 2021-12-19 17:24:25 [INFO] [keep/edge/pkg/edgepublisher/tunnel/edgetunnel.go:70 ] Start a new tunnel connection 2021-12-19 17:24:27 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量... edgepublisher 接收消息 msg: "2021-12-19 17:24:27 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..." -------------------------- 发送云端 msg:model.Message{Header:model.MessageHeader{ID:"51a24512-b0f0-460f-9829-d440bb90b040", ParentID:"", Timestamp:1639905867058, ResourceVersion:"", Sync:false}, Router:model.MessageRoute{Source:"lgent", Group:"/log", Operation:"", Resource:""}, Content:"2021-12-19 17:24:27 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..."} 2021-12-19 17:24:27 [TRAC] [keep/edge/pkg/edgepublisher/publisher/publisher.go:38 ] send message to edge cloud error: websocket session is nil,try write to edgetwin... edtwin 接收消息 msg:"2021-12-19 17:24:27 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..." &{0xc0003fdf20} edgepublisher 接收消息 msg: v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc000260000), Cpu:([]cpu.InfoStat)(0xc00000f788), CpuUsage:4.597701149457331, Mem:(mem.VirtualMemoryStat)(0xc00020c7e0), DiskPartition:([]disk.PartitionStat)(0xc0001269a8), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc000010050), NetIOCountersStat:([]net.IOCountersStat)(0xc000127860), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)} -------------------------- 发送云端 msg:model.Message{Header:model.MessageHeader{ID:"", ParentID:"", Timestamp:0, ResourceVersion:"", Sync:false}, Router:model.MessageRoute{Source:"healthzagent", Group:"", Operation:"", Resource:"Content:v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc000260000), Cpu:([]cpu.InfoStat)(0xc00000f788), CpuUsage:4.597701149457331, Mem:(mem.VirtualMemoryStat)(0xc00020c7e0), DiskPartitionStat:([]disk.PartitionStat)(0xc0001269a8), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc000010050), NetIOCountersStat:([]net.IOCountersStat)(0xc000127860), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)}} 2021-12-19 17:24:28 [TRAC] [keep/edge/pkg/edgepublisher/publisher/publisher.go:38 ] send message to edge cloud error: websocket session is nil,try write to edgetwin... edtwin 接收消息 msg:v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc000260000), Cpu:([]cpu.InfoStat)(0xc00000f788), CpuUsage:4.597701149457331, Mem:(mem.VirtualMemoryStat)(0xc00020c7e0), DiskPartitionStat:([].PartitionStat)(0xc0001269a8), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc000010050), NetIOCountersStat:([]net.IOCountersStat)(0xc000127860), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)} &{0xc0003fdf20} 2021-12-19 17:24:28 [EROR] [keep/edge/pkg/edgepublisher/tunnel/edgetunnel.go:122 ] connect failed: dial tcp 192.168.1.121:20000: connect: no route to host 2021-12-19 17:24:30 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量... edgepublisher 接收消息 msg: "2021-12-19 17:24:30 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..." -------------------------- 发送云端 msg:model.Message{Header:model.MessageHeader{ID:"e2ffa276-6a43-42c9-8bc9-dc7967a32141", ParentID:"", Timestamp:1639905870066, ResourceVersion:"", Sync:false}, Router:model.MessageRoute{Source:"lgent", Group:"/log", Operation:"", Resource:""}, Content:"2021-12-19 17:24:30 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..."} 2021-12-19 17:24:30 [TRAC] [keep/edge/pkg/edgepublisher/publisher/publisher.go:38 ] send message to edge cloud error: websocket session is nil,try write to edgetwin... edgepublisher 接收消息 msg: v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc0001a6180), Cpu:([]cpu.InfoStat)(0xc0002902b8), CpuUsage:8.18414322259908, Mem:(mem.VirtualMemoryStat)(0xc000162000), DiskPartitionS([]disk.PartitionStat)(0xc000291dd0), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc00029a068), NetIOCountersStat:([]net.IOCountersStat)(0xc0003b0c00), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)} 2021-12-19 17:24:33 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量... edgepublisher 接收消息 msg: "2021-12-19 17:24:33 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..." 2021-12-19 17:24:33 [INFO] [keep/edge/pkg/edgepublisher/tunnel/edgetunnel.go:70 ] Start a new tunnel connection edgepublisher 接收消息 msg: v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc00035c000), Cpu:([]cpu.InfoStat)(0xc000127c68), CpuUsage:1.1320754717065746, Mem:(mem.VirtualMemoryStat)(0xc00020c900), DiskPartitiot:([]disk.PartitionStat)(0xc000339710), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc0000100e8), NetIOCountersStat:([]net.IOCountersStat)(0xc0002725a0), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)} 2021-12-19 17:24:36 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量... edgepublisher 接收消息 msg: "2021-12-19 17:24:36 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量..." 2021-12-19 17:24:36 [EROR] [keep/edge/pkg/edgepublisher/tunnel/edgetunnel.go:122 ] connect failed: dial tcp 192.168.1.121:20000: connect: no route to host edgepublisher 接收消息 msg: v1alpha1.HealthzAgent{Enable:false, HostInfoStat:(host.InfoStat)(0xc0001a6240), Cpu:([]cpu.InfoStat)(0xc0003b0fa8), CpuUsage:0.6297229219508278, Mem:(mem.VirtualMemoryStat)(0xc000162120), DiskPartitiot:([]disk.PartitionStat)(0xc0003b4a20), DiskIOCountersStat:(map[string]disk.IOCountersStat)(0xc00029a140), NetIOCountersStat:([]net.IOCountersStat)(0xc0003b5848), DefaultEdgeHealthInterval:0, DeviceMqttTopics:[]string(nil)} 2021-12-19 17:24:39 [DEBG] [/root/go/pkg/mod/github.com/robfig/cron@v1.2.0/cron.go:92 ] 查询节点用量...