search

greatcare / pm2-zabbix

A Node.js PM2 monitoring tool for Zabbix.
MIT License
87 stars 60 forks source link

Error Code 2 #55

Open ka-may opened 2 years ago

ka-may commented 2 years ago

Hi, I wanted to get up-to-date data about a running process in PM2 using zabbix. When trying to run pm2-zabbix –monitor:

"name":"pm2-zabbix","hostname":"node","pid":2440,"level":50,"event":"PM2ZabbixMonitor#processListSendingError","error":{"killed":false ,"code":2,"signal":null,"cmd":"/usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -"},"msg":"Failed to send process list with stats to server: Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -\n\n at ChildProcess.exithandler (child_process.js: 308:12)\n at ChildProcess.emit (events.js:314:20)\n at maybeClose (internal/child_process.js:1022:16)\n at Socket.<anonymous> (internal/child_process.js:444 :11)\n at Socket.emit (events.js:314:20)\n at Pipe.<anonymous> (net.js:676:12) {\n killed: false,\n code: 2,\n signal: null,\n cmd: '/usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -'\n}","time":"2022-08-19T09:28 :27.553Z","v":0}

When trying to run LOG_LEVEL=trace pm2-zabbix –monitor:

{"name":"pm2-zabbix","hostname":"node","pid":2492,"level":30,"event":"PM2ZabbixMonitor#starting","monitor":true,"msg":"Starting PM2ZabbixMonitor (monitoring enabled: true)","time":"2022-08-19T09:45:49.918Z","v":0}
{"name":"pm2-zabbix","hostname":"node","pid":2492,"level":20,"event":"PM2ZabbixMonitor#processMapGenerated","processMap":{"hello-0":{"name":"hello","status":"online","resources":{"memory":53964800,"cpu":0},"restarts":0}},"msg":"Process map generated","time":"2022-08-19T09:45:49.955Z","v":0}
{"name":"pm2-zabbix","hostname":"node","pid":2492,"level":30,"event":"PM2ZabbixMonitor#started","msg":"PM2ZabbixMonitor started","time":"2022-08-19T09:45:49.956Z","v":0}
{"name":"pm2-zabbix","hostname":"node","pid":2492,"level":10,"event":"PM2ZabbixMonitor#gotProcessMap","processMap":{"hello-0":{"name":"hello","status":"online","resources":{"memory":53964800,"cpu":0},"restarts":0}},"msg":"","time":"2022-08-19T09:46:04.967Z","v":0}
{"name":"pm2-zabbix","hostname":"node","pid":2492,"level":20,"event":"PM2ZabbixMonitor#sendProcessList","processList":{"pm2.processes[hello-0,status]":"online","pm2.processes[hello-0,cpu]":0,"pm2.processes[hello-0,memory]":53964800,"pm2.processes[hello-0,restarts]":0},"msg":"Sending process list with stats","time":"2022-08-19T09:46:04.967Z","v":0}
{"name":"pm2-zabbix","hostname":"node","pid":2492,"level":10,"event":"ZabbixSender#send","bin":"/usr/bin/zabbix_sender","args":["--config","/etc/zabbix/zabbix_agentd.conf","--input-file","-"],"stdIn":"node pm2.processes[hello-0,status] online\nnode pm2.processes[hello-0,cpu] 0\nnode pm2.processes[hello-0,memory] 53964800\nnode pm2.processes[hello-0,restarts] 0\n","debug":false,"msg":"Executing: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -","time":"2022-08-19T09:46:04.969Z","v":0}
{"name":"pm2-zabbix","hostname":"node","pid":2492,"level":50,"event":"PM2ZabbixMonitor#processListSendingError","error":{"killed":false,"code":2,"signal":null,"cmd":"/usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -"},"msg":"Failed to send process list with stats to server: Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -\n\n    at ChildProcess.exithandler (child_process.js:308:12)\n    at ChildProcess.emit (events.js:314:20)\n    at maybeClose (internal/child_process.js:1022:16)\n    at Socket.<anonymous> (internal/child_process.js:444:11)\n    at Socket.emit (events.js:314:20)\n    at Pipe.<anonymous> (net.js:676:12) {\n  killed: false,\n  code: 2,\n  signal: null,\n  cmd: '/usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -'\n}","time":"2022-08-19T09:46:04.994Z","v":0}
{"name":"pm2-zabbix","hostname":"node","pid":2492,"level":10,"event":"PM2ZabbixMonitor#gotPM2State","processState":{"name":"PM2","status":"online","resources":{"cpu":0.2371246925837011,"memory":61444096},"restarts":0,"pid":1131},"msg":"","time":"2022-08-19T09:46:05.003Z","v":0}
{"name":"pm2-zabbix","hostname":"node","pid":2492,"level":20,"event":"PM2ZabbixMonitor#sendPM2Status","status":{"pm2.status":"online","pm2.cpu":0.2371246925837011,"pm2.memory":61444096,"pm2.pid":1131},"msg":"Sending PM2 status","time":"2022-08-19T09:46:05.004Z","v":0}
{"name":"pm2-zabbix","hostname":"node","pid":2492,"level":10,"event":"ZabbixSender#send","bin":"/usr/bin/zabbix_sender","args":["--config","/etc/zabbix/zabbix_agentd.conf","--input-file","-"],"stdIn":"node pm2.status online\nnode pm2.cpu 0\nnode pm2.memory 61444096\nnode pm2.pid 1131\n","debug":false,"msg":"Executing: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -","time":"2022-08-19T09:46:05.005Z","v":0}
{"name":"pm2-zabbix","hostname":"node","pid":2492,"level":20,"event":"PM2ZabbixMonitor#PM2StatusSent","msg":"PM2 status sent","time":"2022-08-19T09:46:05.020Z","v":0}

i studied this https://github.com/greatcare/pm2-zabbix/blob/master/Troubleshooting.md

  1. pm2-daemon was successfully discovered in zabbix, while there is no information about the child process pm2 --discovery successfully detects running processes. image

  2. The hostname on the node is the same as the hostname in zabbix-agent.conf and is the same as the node on the zabbix server

  3. I can successfully send data using zabbix-sender

Resources used: Ubuntu server 22, NPM 8.5.1, PM2 5.2.0, PM2-zabbix 0.3.2.

I will appreciate any help, thanks

ka-may commented 2 years ago

Zabbix-Server logs: 

2357495:20220819:132837.014 trapper got '{"request":"sender data","data":[{"host":"node","key":"pm2.processes[hello-0,status]","value":"online"},{"host":"node","key":"pm2.processes[hello-0,cpu]","value":"0"},{"host":"node","key":"pm2.processes[hello-0,memory]","value":"53968896"},{"host":"node","key":"pm2.processes[hello-0,restarts]","value":"0"}]}'
2357495:20220819:132837.014 cannot retrieve key "pm2.processes[hello-0,status]" on host "node" from configuration cache
2357495:20220819:132837.014 cannot retrieve key "pm2.processes[hello-0,cpu]" on host "node" from configuration cache
2357495:20220819:132837.014 cannot retrieve key "pm2.processes[hello-0,memory]" on host "node" from configuration cache
2357495:20220819:132837.014 cannot retrieve key "pm2.processes[hello-0,restarts]" on host "node" from configuration cache
2357492:20220819:132837.018 trapper got '{"request":"sender data","data":[{"host":"node","key":"pm2.status","value":"online"},{"host":"node","key":"pm2.cpu","value":"0"},{"host":"node","key":"pm2.memory","value":"61538304"},{"host":"node","key":"pm2.pid","value":"1131"}]}'
toontoet commented 1 year ago

What version of zabbix_sender are you using? I'm running into the same kind of issue and notices that it work using zabbix_sender 3.4.15 and it stopped working when I upgraded to 6.0.7.

toontoet commented 1 year ago

After some debugging it worked for me when I added the line Defaults:zabbix !requiretty, always_set_home to the sudoers file. The older version didn't need this.

fbnielsen1959 commented 9 months ago

I have discovered a bug/feature in monitor.js that causes this error. HOSTNAME for Zabbix is NEVER taken from zabbix-agent.conf, but either from --HOSTNAME="hostname" or from os.hostname(); (se line 40) A simple fix that forces zabbix_sender to use HOSTNAME from zabbix-agent.conf: pm2-zabbix --monitor --hostname='-'

systemd / sysv will have to be modified also