Open Fchen48 opened 6 years ago
There seem to be two separate issues here:
LOG_LEVEL=trace pm2-zabbix --monitor
and observe what it's sending to the server. Usually, the keys will not exist on the server (missing LLD configuration step) or the hostname will be wrong.npm install -g pm2-zabbix
installs packages in the system's global node_modules, but some Node version managers like nvm
can override your npm config and cause it to install globals elsewhere. Make sure you perform a real global install, not a local one for your user. If unable, you will need to change the unit file to match your custom paths for Node.js and the script.I rechecked the installation but doesn't found anything unusual.
root@app01 ~ # npm i -g pm2-zabbix
/usr/local/bin/pm2-zabbix -> /usr/local/lib/node_modules/pm2-zabbix/monitor.js
+ pm2-zabbix@0.3.2
updated 2 packages in 3.373s
Here is the trace log:
root@app01 ~ # LOG_LEVEL=trace pm2-zabbix --monitor
{"name":"pm2-zabbix","hostname":"app01","pid":13571,"level":30,"event":"PM2ZabbixMonitor#starting","monitor":true,"msg":"Starting PM2ZabbixMonitor (monitoring enabled: true)","time":"2018-04-12T13:42:52.675Z","v":0}
{"name":"pm2-zabbix","hostname":"app01","pid":13571,"level":20,"event":"PM2ZabbixMonitor#processMapGenerated","processMap":{"pm2-hook-1":{"name":"pm2-hook","status":"online","resources":{"memory":52666368,"cpu":0},"restarts":0},"EPKW-4 {"name":"EPKW","status":"stopped","resources":{"memory":0,"cpu":0},"restarts":4}},"msg":"Process map generated","time":"2018-04-12T13:42:52.690Z","v":0}
{"name":"pm2-zabbix","hostname":"app01","pid":13571,"level":30,"event":"PM2ZabbixMonitor#started","msg":"PM2ZabbixMonitor started","time":"2018-04-12T13:42:52.690Z","v":0}
{"name":"pm2-zabbix","hostname":"app01","pid":13571,"level":10,"event":"PM2ZabbixMonitor#gotProcessMap","processMap":{"pm2-hook-1":{"name":"pm2-hook","status":"online","resources":{"memory":51863552,"cpu":0},"restarts":0},"EPKW-4":{"name":"EPKW","status":"stopped","resources":{"memory":0,"cpu":0},"restarts":4}},"msg":"","time":"2018-04-12T13:43:07.707Z","v":0}
{"name":"pm2-zabbix","hostname":"app01","pid":13571,"level":20,"event":"PM2ZabbixMonitor#sendProcessList","processList":{"pm2.processes[pm2-hook-1,status]":"online","pm2.processes[pm2-hook-1,cpu]":0,"pm2.processes[pm2-hook-1,memory]":51863552,"pm2.processes[pm2-hook-1,restarts]":0,"pm2.processes[EPKW-4,status]":"stopped","pm2.processes[EPKW-4,cpu]":0,"pm2.processes[EPKW-4,memory]":0,"pm2.processes[EPKW-4,restarts]":4},"msg":"Sending process list with stats","time":"2018-04-12T13:43:07.707Z","v":0}
{"name":"pm2-zabbix","hostname":"app01","pid":13571,"level":10,"event":"ZabbixSender#send","bin":"/usr/bin/zabbix_sender","args":["--config","/etc/zabbix/zabbix_agentd.conf","--input-file","-"],"stdIn":"app01 pm2.processes[pm2-hook-1,status] online\napp01 pm2.processes[pm2-hook-1,cpu] 0\napp01 pm2.processes[pm2-hook-1,memory] 51863552\napp01 pm2.processes[pm2-hook-1,restarts] 0\napp01 pm2.processes[EPKW-4,status] stopped\napp01 pm2.processes[EPKW-4,cpu] 0\napp01 pm2.processes[EPKW-4,memory] 0\napp01 pm2.processes[EPKW-4,restarts] 4\n","debug":false,"msg":"Executing: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -","time":"2018-04-12T13:43:07.707Z","v":0}
{"name":"pm2-zabbix","hostname":"app01","pid":13571,"level":50,"event":"PM2ZabbixMonitor#processListSendingError","error":{"killed":false,"code":2,"signal":null,"cmd":"/usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -"},"msg":"Failed to send process list with stats to server: Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -\n","time":"2018-04-12T13:43:07.715Z","v":0}
{"name":"pm2-zabbix","hostname":"app01","pid":13571,"level":10,"event":"PM2ZabbixMonitor#gotPM2State","processState":{"name":"PM2","status":"online","resources":{"cpu":0.036499984644320554,"memory":60280832},"restarts":0,"pid":8510},"msg":"","time":"2018-04-12T13:43:07.719Z","v":0}
{"name":"pm2-zabbix","hostname":"app01","pid":13571,"level":20,"event":"PM2ZabbixMonitor#sendPM2Status","status":{"pm2.status":"online","pm2.cpu":0.036499984644320554,"pm2.memory":60280832,"pm2.pid":8510},"msg":"Sending PM2 status","time":"2018-04-12T13:43:07.719Z","v":0}
{"name":"pm2-zabbix","hostname":"app01","pid":13571,"level":10,"event":"ZabbixSender#send","bin":"/usr/bin/zabbix_sender","args":["--config","/etc/zabbix/zabbix_agentd.conf","--input-file","-"],"stdIn":"app01 pm2.status online\napp01 pm2.cpu 0\napp01 pm2.memory 60280832\napp01 pm2.pid 8510\n","debug":false,"msg":"Executing: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -","time":"2018-04-12T13:43:07.719Z","v":0}
{"name":"pm2-zabbix","hostname":"app01","pid":13571,"level":50,"event":"PM2ZabbixMonitor#PM2StatusSent","error":{"killed":false,"code":2,"signal":null,"cmd":"/usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -"},"msg":"Failed to send PM2 status: Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -\n","time":"2018-04-12T13:43:07.721Z","v":0}
As for the systemd issue, it seems that status=217/USER
is commonly caused by a missing or malformed user specification. Note this section in the default unit file we provide:
[Service]
Type=simple
User=debian
Clearly, if user debian does not exist on your system, the service must fail to start. You will need to change the user to the one that you intend to run your PM2 under.
Now, the trace log shows you which keys are being sent to the Zabbix server. Can you confirm that those keys exist in the Zabbix server management GUI? The line with ZabbixSender#send
contains the complete text that is being sent to zabbix_sender in the stdIn field, so you should be able to see whether the host name and the key names match.
Hi,
I changed the user to zabbix but no I get another error, pm2-zabbix will create the folder nonexistent under / but have no permission for it.
Apr 14 11:43:00 app01 sudo[3244]: zabbix : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/pm2-zabbix --discover
Apr 14 11:43:00 app01 sudo[3244]: pam_unix(sudo:session): session opened for user root by (uid=0)
Apr 14 11:43:00 app01 sudo[3244]: pam_unix(sudo:session): session closed for user root
Apr 14 11:43:10 app01 systemd[1]: Stopped PM2 monitor for Zabbix.
Apr 14 11:43:19 app01 systemd[1]: Failed to reset devices.list on /system.slice/pm2-zabbix.service: Operation not permitted
Apr 14 11:43:19 app01 systemd[1]: Started PM2 monitor for Zabbix.
Apr 14 11:43:19 app01 pm2-zabbix[3260]: Error: EACCES: permission denied, mkdir '/nonexistent'
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.fs.mkdirSync (fs.js:902:18)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at sync (/usr/local/lib/node_modules/pm2-zabbix/node_modules/mkdirp/index.js:71:13)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at sync (/usr/local/lib/node_modules/pm2-zabbix/node_modules/mkdirp/index.js:77:24)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Function.sync (/usr/local/lib/node_modules/pm2-zabbix/node_modules/mkdirp/index.js:77:24)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at module.exports.Client.initFileStructure (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/Client.js:132:14)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at new module.exports (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/Client.js:38:8)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at new module.exports (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/API.js:99:17)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.<anonymous> (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/index.js:11:18)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Module._compile (internal/modules/cjs/loader.js:654:30)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.Module._extensions..js (internal/modules/cjs/loader.js:665:10)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: Error: EACCES: permission denied, mkdir '/nonexistent'
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.fs.mkdirSync (fs.js:902:18)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at sync (/usr/local/lib/node_modules/pm2-zabbix/node_modules/mkdirp/index.js:71:13)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at sync (/usr/local/lib/node_modules/pm2-zabbix/node_modules/mkdirp/index.js:77:24)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Function.sync (/usr/local/lib/node_modules/pm2-zabbix/node_modules/mkdirp/index.js:77:24)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at module.exports.Client.initFileStructure (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/Client.js:140:14)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at new module.exports (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/Client.js:38:8)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at new module.exports (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/API.js:99:17)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.<anonymous> (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/index.js:11:18)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Module._compile (internal/modules/cjs/loader.js:654:30)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.Module._extensions..js (internal/modules/cjs/loader.js:665:10)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: Error: ENOENT: no such file or directory, open '/nonexistent/.pm2/module_conf.json'
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.fs.openSync (fs.js:660:18)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.fs.writeFileSync (fs.js:1319:33)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at module.exports.Client.initFileStructure (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/Client.js:148:10)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at new module.exports (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/Client.js:38:8)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at new module.exports (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/API.js:99:17)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.<anonymous> (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/index.js:11:18)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Module._compile (internal/modules/cjs/loader.js:654:30)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.Module._extensions..js (internal/modules/cjs/loader.js:665:10)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Module.load (internal/modules/cjs/loader.js:566:32)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at tryModuleLoad (internal/modules/cjs/loader.js:506:12)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: Error: EACCES: permission denied, mkdir '/nonexistent'
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.fs.mkdirSync (fs.js:902:18)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at sync (/usr/local/lib/node_modules/pm2-zabbix/node_modules/mkdirp/index.js:71:13)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at sync (/usr/local/lib/node_modules/pm2-zabbix/node_modules/mkdirp/index.js:77:24)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Function.sync (/usr/local/lib/node_modules/pm2-zabbix/node_modules/mkdirp/index.js:77:24)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at module.exports.Client.initFileStructure (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/Client.js:156:14)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at new module.exports (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/Client.js:38:8)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at new module.exports (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/API.js:99:17)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.<anonymous> (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/index.js:11:18)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Module._compile (internal/modules/cjs/loader.js:654:30)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.Module._extensions..js (internal/modules/cjs/loader.js:665:10)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: fs.js:660
Apr 14 11:43:19 app01 pm2-zabbix[3260]: return binding.open(pathModule.toNamespacedPath(path),
Apr 14 11:43:19 app01 pm2-zabbix[3260]: ^
Apr 14 11:43:19 app01 pm2-zabbix[3260]: Error: ENOENT: no such file or directory, open '/nonexistent/.pm2/pm2.log'
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at Object.fs.openSync (fs.js:660:18)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at module.exports.Client.launchDaemon (/usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/Client.js:219:14)
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at /usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/Client.js:103:10
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at /usr/local/lib/node_modules/pm2-zabbix/node_modules/pm2/lib/Client.js:307:14
Apr 14 11:43:19 app01 pm2-zabbix[3260]: at process._tickCallback (internal/process/next_tick.js:176:11)
Apr 14 11:43:19 app01 systemd[1]: pm2-zabbix.service: Main process exited, code=exited, status=1/FAILURE
and for the other problem, zabbix_sender
send only the hostname app01
and not the FQDN which is set in the zabbix config file and in the zabbix server configuration. Is there a way that the hostname will be taken from the zabbix configuration instead of /etc/hostname
, because I'm not able to change it, Proxmox will reset it on container start.
I seem to be having this problem also. I have had it working in the past and am not sure when it stopped.
pm2-zabbix --monitor
{"name":"pm2-zabbix","hostname":"cameras01","pid":1479,"level":50,"event":"PM2ZabbixMonitor#PM2StatusSent","error":{"killed":false,"code":2,"signal":null,"cmd":"/usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -"},"msg":"Failed to send PM2 status: Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -\n","time":"2018-10-21T23:25:00.934Z","v":0}
can any one help me out i am still unable to receive any notfication on zabbix related to pm2 status
{"name":"pm2-zabbix","hostname":"ip-172-31-43-89","pid":25685,"level":50,"event":"PM2ZabbixMonitor#processListSendingError","error":{"killed":false,"code":1,"signal":null,"cmd":"/usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -"},"msg":"Failed to send process list with stats to server: Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -\n","time":"2018-11-04T22:02:01.473Z","v":0} {"name":"pm2-zabbix","hostname":"ip-172-31-43-89","pid":25685,"level":50,"event":"PM2ZabbixMonitor#PM2StatusSent","error":{"killed":false,"code":1,"signal":null,"cmd":"/usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -"},"msg":"Failed to send PM2 status: Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -\n","time":"2018-11-04T22:02:01.473Z","v":0} {"name":"pm2-zabbix","hostname":"ip-172-31-43-89","pid":25685,"level":10,"event":"PM2ZabbixMonitor#gotPM2State","processState":{"name":"PM2","status":"online","resources":{"cpu":0,"memory":48480256},"restarts":0,"pid":13550},"msg":"","time":"2018-11-04T22:02:01.474Z","v":0}
i tried pm2 stop cron but didnt receive any notification.
^C
root@ip-172-31-43-89:/lib/systemd/system# pm2 stop cron
[PM2] Applying action stopProcessId on app [cron](ids: 0)
[PM2] cron ✓
┌──────────┬────┬─────────┬──────┬─────┬─────────┬─────────┬────────┬─────┬────────┬──────┬──────────┐
│ App name │ id │ version │ mode │ pid │ status │ restart │ uptime │ cpu │ mem │ user │ watching │
├──────────┼────┼─────────┼──────┼─────┼─────────┼─────────┼────────┼─────┼────────┼──────┼──────────┤
│ cron │ 0 │ N/A │ fork │ 0 │ stopped │ 60 │ 0 │ 0% │ 0 B │ root │ disabled │
└──────────┴────┴─────────┴──────┴─────┴─────────┴─────────┴────────┴─────┴────────┴──────┴──────────┘
Use pm2 show <id|name>
to get more details about an app
Hi Im having the same problem, if tried everthing i can find on google but nuthing helps..
I made a replica of two of my servers (nginx + varnish and nodejs), on the original server everthing is working fine with zabbix, however on my replicas i'm havig problems with pm2-zabbix and zabbix-varnish but no problem with system info or nginx info.
This is the error i get wen i run (on the node server) "sudo pm2-zabbix --monitor --debug": {"name":"pm2-zabbix","hostname":"nodejs61replica","pid":98546,"level":50,"event":"PM2ZabbixMonitor#PM2StatusSent","error":{"kilt-file -"},"msg":"Failed to send PM2 status: Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.c {"name":"pm2-zabbix","hostname":"nodejs61replica","pid":98546,"level":50,"event":"PM2ZabbixMonitor#PM2StatusSent","error":{"killed":false,"code":1,"signal":null,"cmd":"/usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -"},"msg":"Failed to send PM2 status: Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -\n","time":"2020-05-03T10:10:03.944Z","v":0}
on my varnish server i run "sudo /usr/local/bin/zabbix-varnish-cache.py -i '' send -c /etc/zabbix/zabbix_agentd.conf" and it seems evrthing is OK but zabbix server dos not recive the info:
Help please!
Hi,
I installed pm2-zabbix on a fresh Ubuntu 16.04 LTS LXC container (privileged), after install I tried
pm2-zabbix --monitor
using user root and I receiveThe same error appears if I will perform the command with user zabbix.
Next error I receive is if I will start pm2-zabbix using systemd
systemctl status pm2-zabbix
PM2 is running under user root and sudoers file is installed.
Sending with
zabbix_sender
works fine!Anything else I can check?