search

bytedance / Elkeid

Elkeid is an open source solution that can meet the security requirements of various workloads such as hosts, containers and K8s, and serverless. It is derived from ByteDance's internal best practices.
https://elkeid.bytedance.com
2.23k stars 429 forks source link

使用elkeidup进行完整部署V1.9.1.3 后agent报错 #474

Closed aiwennba closed 1 year ago

aiwennba commented 1 year ago

2023-03-27T14:03:57.511+0800 INFO plugin/plugin.go:211 plugin has been loaded {"plugin": "baseline", "pver": "1.0.1.23", "psign": "1d86d4e7dd22e5d227934f05eb9f1d0210d86924f02db53c06edb09dccc72a3e"} 2023-03-27T14:03:57.511+0800 INFO plugin/plugin.go:227 sync done 2023-03-27T14:03:57.511+0800 INFO plugin/plugin.go:181 context has been canceled, will shutdown all plugins 2023-03-27T14:03:57.511+0800 INFO plugin/plugin_linux.go:33 plugin is running, will shutdown it {"plugin": "scanner", "pver": "3.1.9.6", "psign": "527c6ea0caac3b0604021de5aa2d34e4b9fae715e5e6cdd37e8f485869f923c2"} 2023-03-27T14:03:57.512+0800 INFO plugin/plugin_linux.go:160 gorountine of receiving plugin's data will exit {"plugin": "scanner", "pver": "3.1.9.6", "psign": "527c6ea0caac3b0604021de5aa2d34e4b9fae715e5e6cdd37e8f485869f923c2"} 2023-03-27T14:03:57.512+0800 INFO plugin/plugin_linux.go:33 plugin is running, will shutdown it {"plugin": "collector", "pver": "1.0.0.140", "psign": "728e0d487317d0fbc63811e50c302fa024fd80fefab23c03113d4437a22bcb3d"} 2023-03-27T14:03:57.512+0800 INFO plugin/plugin_linux.go:160 gorountine of receiving plugin's data will exit {"plugin": "collector", "pver": "1.0.0.140", "psign": "728e0d487317d0fbc63811e50c302fa024fd80fefab23c03113d4437a22bcb3d"} 2023-03-27T14:03:57.512+0800 INFO plugin/plugin_linux.go:33 plugin is running, will shutdown it {"plugin": "journal_watcher", "pver": "1.0.0.23", "psign": "d8dd6d549f6af203c4e1eb79b34a3477214a7cf314559ef79cf141b81461c25b"} 2023-03-27T14:03:57.512+0800 INFO plugin/plugin_linux.go:160 gorountine of receiving plugin's data will exit {"plugin": "journal_watcher", "pver": "1.0.0.23", "psign": "d8dd6d549f6af203c4e1eb79b34a3477214a7cf314559ef79cf141b81461c25b"} 2023-03-27T14:03:57.512+0800 INFO plugin/plugin_linux.go:33 plugin is running, will shutdown it {"plugin": "rasp", "pver": "1.9.1.44", "psign": "5768cb497d5438bfbf5ae40abf9172295eb5e5ff633eece240e30dd81de82b56"} 2023-03-27T14:03:57.512+0800 INFO plugin/plugin_linux.go:160 gorountine of receiving plugin's data will exit {"plugin": "rasp", "pver": "1.9.1.44", "psign": "5768cb497d5438bfbf5ae40abf9172295eb5e5ff633eece240e30dd81de82b56"} 2023-03-27T14:03:57.512+0800 INFO plugin/plugin_linux.go:33 plugin is running, will shutdown it {"plugin": "baseline", "pver": "1.0.1.23", "psign": "1d86d4e7dd22e5d227934f05eb9f1d0210d86924f02db53c06edb09dccc72a3e"} 2023-03-27T14:03:57.512+0800 INFO plugin/plugin_linux.go:160 gorountine of receiving plugin's data will exit {"plugin": "baseline", "pver": "1.0.1.23", "psign": "1d86d4e7dd22e5d227934f05eb9f1d0210d86924f02db53c06edb09dccc72a3e"} 2023-03-27T14:03:57.611+0800 INFO plugin/plugin_linux.go:136 plugin has exited with code 0 {"plugin": "journal_watcher", "pver": "1.0.0.23", "psign": "d8dd6d549f6af203c4e1eb79b34a3477214a7cf314559ef79cf141b81461c25b"} 2023-03-27T14:03:57.611+0800 INFO plugin/plugin_linux.go:136 plugin has exited with code 0 {"plugin": "collector", "pver": "1.0.0.140", "psign": "728e0d487317d0fbc63811e50c302fa024fd80fefab23c03113d4437a22bcb3d"} 2023-03-27T14:03:57.611+0800 INFO plugin/plugin_linux.go:136 plugin has exited with code 0 {"plugin": "scanner", "pver": "3.1.9.6", "psign": "527c6ea0caac3b0604021de5aa2d34e4b9fae715e5e6cdd37e8f485869f923c2"} 2023-03-27T14:03:57.709+0800 INFO plugin/plugin_linux.go:142 gorountine of waiting plugin's process will exit {"plugin": "scanner", "pver": "3.1.9.6", "psign": "527c6ea0caac3b0604021de5aa2d34e4b9fae715e5e6cdd37e8f485869f923c2"} 2023-03-27T14:03:57.709+0800 INFO plugin/plugin_linux.go:43 plugin has been shutdown gracefully {"plugin": "collector", "pver": "1.0.0.140", "psign": "728e0d487317d0fbc63811e50c302fa024fd80fefab23c03113d4437a22bcb3d"} 2023-03-27T14:03:57.611+0800 INFO plugin/plugin_linux.go:43 plugin has been shutdown gracefully {"plugin": "journal_watcher", "pver": "1.0.0.23", "psign": "d8dd6d549f6af203c4e1eb79b34a3477214a7cf314559ef79cf141b81461c25b"} 2023-03-27T14:03:57.611+0800 INFO plugin/plugin_linux.go:167 gorountine of sending task to plugin will exit {"plugin": "journal_watcher", "pver": "1.0.0.23", "psign": "d8dd6d549f6af203c4e1eb79b34a3477214a7cf314559ef79cf141b81461c25b"} 2023-03-27T14:03:57.709+0800 INFO plugin/plugin_linux.go:142 gorountine of waiting plugin's process will exit {"plugin": "collector", "pver": "1.0.0.140", "psign": "728e0d487317d0fbc63811e50c302fa024fd80fefab23c03113d4437a22bcb3d"} 2023-03-27T14:03:57.709+0800 INFO plugin/plugin_linux.go:167 gorountine of sending task to plugin will exit {"plugin": "scanner", "pver": "3.1.9.6", "psign": "527c6ea0caac3b0604021de5aa2d34e4b9fae715e5e6cdd37e8f485869f923c2"} 2023-03-27T14:03:57.710+0800 INFO plugin/plugin_linux.go:43 plugin has been shutdown gracefully {"plugin": "scanner", "pver": "3.1.9.6", "psign": "527c6ea0caac3b0604021de5aa2d34e4b9fae715e5e6cdd37e8f485869f923c2"} 2023-03-27T14:03:57.611+0800 INFO plugin/plugin_linux.go:142 gorountine of waiting plugin's process will exit {"plugin": "journal_watcher", "pver": "1.0.0.23", "psign": "d8dd6d549f6af203c4e1eb79b34a3477214a7cf314559ef79cf141b81461c25b"} 2023-03-27T14:03:57.710+0800 INFO plugin/plugin_linux.go:167 gorountine of sending task to plugin will exit {"plugin": "collector", "pver": "1.0.0.140", "psign": "728e0d487317d0fbc63811e50c302fa024fd80fefab23c03113d4437a22bcb3d"} 2023-03-27T14:04:07.521+0800 WARN plugin/plugin_linux.go:38 because of plugin exit's timeout, will kill it {"plugin": "rasp", "pver": "1.9.1.44", "psign": "5768cb497d5438bfbf5ae40abf9172295eb5e5ff633eece240e30dd81de82b56"} 2023-03-27T14:04:07.521+0800 WARN plugin/plugin_linux.go:38 because of plugin exit's timeout, will kill it {"plugin": "baseline", "pver": "1.0.1.23", "psign": "1d86d4e7dd22e5d227934f05eb9f1d0210d86924f02db53c06edb09dccc72a3e"} 2023-03-27T14:04:07.523+0800 ERROR plugin/plugin_linux.go:134 plugin has exited with error: signal: killed, code: -1 {"plugin": "baseline", "pver": "1.0.1.23", "psign": "1d86d4e7dd22e5d227934f05eb9f1d0210d86924f02db53c06edb09dccc72a3e"} 2023-03-27T14:04:07.523+0800 INFO plugin/plugin_linux.go:142 gorountine of waiting plugin's process will exit {"plugin": "baseline", "pver": "1.0.1.23", "psign": "1d86d4e7dd22e5d227934f05eb9f1d0210d86924f02db53c06edb09dccc72a3e"} 2023-03-27T14:04:07.523+0800 INFO plugin/plugin_linux.go:41 plugin has been killed {"plugin": "baseline", "pver": "1.0.1.23", "psign": "1d86d4e7dd22e5d227934f05eb9f1d0210d86924f02db53c06edb09dccc72a3e"} 2023-03-27T14:04:07.523+0800 INFO plugin/plugin_linux.go:167 gorountine of sending task to plugin will exit {"plugin": "baseline", "pver": "1.0.1.23", "psign": "1d86d4e7dd22e5d227934f05eb9f1d0210d86924f02db53c06edb09dccc72a3e"} 2023-03-27T14:04:07.523+0800 ERROR plugin/plugin_linux.go:134 plugin has exited with error: signal: killed, code: -1 {"plugin": "rasp", "pver": "1.9.1.44", "psign": "5768cb497d5438bfbf5ae40abf9172295eb5e5ff633eece240e30dd81de82b56"} 2023-03-27T14:04:07.523+0800 INFO plugin/plugin_linux.go:142 gorountine of waiting plugin's process will exit {"plugin": "rasp", "pver": "1.9.1.44", "psign": "5768cb497d5438bfbf5ae40abf9172295eb5e5ff633eece240e30dd81de82b56"} 2023-03-27T14:04:07.523+0800 INFO plugin/plugin_linux.go:41 plugin has been killed {"plugin": "rasp", "pver": "1.9.1.44", "psign": "5768cb497d5438bfbf5ae40abf9172295eb5e5ff633eece240e30dd81de82b56"} 2023-03-27T14:04:07.523+0800 INFO plugin/plugin_linux.go:167 gorountine of sending task to plugin will exit {"plugin": "rasp", "pver": "1.9.1.44", "psign": "5768cb497d5438bfbf5ae40abf9172295eb5e5ff633eece240e30dd81de82b56"}

TaeRoen commented 1 year ago

这部分log只是agent正常退出，逐个kill plugin的log，log里看不到退出的原因。 可以先执行 /etc/elkeid/elkeidctl status 看下agent启动了多长时间，是否是一直在重启。 同时也执行下 systemctl status elkeid_ac ，看下ac是否也在重启，ac重启也会导致agent被动重启。