greatcare / pm2-zabbix

A Node.js PM2 monitoring tool for Zabbix.
MIT License
88 stars 60 forks source link

get help, [ ERR ] Periodic sending of PM2 status failed: { [Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file - #2

Closed eg8320 closed 6 years ago

eg8320 commented 8 years ago

grep -ri pm2.processes /etc/zabbix/zabbix_agentd.conf

UserParameter=pm2.processes,sudo -u root /usr/local/node/bin/pm2-zabbix --discover

[root@upelk2 ~]# pm2-zabbix --discover

{ "data": [ { "{#PROCESS_ID}": "www-1", "{#PROCESS_NAME}": "www" }, { "{#PROCESS_ID}": "pm2-http-interface-2", "{#PROCESS_NAME}": "pm2-http-interface" } ] }[root@upelk2 ~]#

[root@upelk2 ~]# pm2-zabbix --monitor --debug

zabbix server is no pm2-zabbix graph

rkaw92 commented 8 years ago

Hi. Has the discovery process completed successfully? The exit code of zabbix_sender was 2, which implies that the data was sent, but the server has failed or refused to process it. This could happen if the server has not discovered the processes, and it has no item for your host to attribute the data to.

Can you please go to the host's settings page in Zabbix and verify that the items have been created? The relevant items can be seen at Configuration → Hosts → <your host> → Items. If the items are not there, that means discovery has failed or never been attempted and you should check if the Zabbix Agent is running on the host and not generating errors (/var/log/zabbix/zabbix_agentd.log on Debian systems).

eg8320 commented 8 years ago

thanks for you help, I check Host item: PM2 Daemon and PM2 Processes items is existed by discovery process ,but zabbix-sever don't received any data。 don't find any error in /var/log/zabbix/zabbix_agentd.log.

pm2-zabbix --monitor --debug

[ ERR ] Periodic sending of PM2 status failed: { [Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file - ] killed: false, code: 2, signal: null, cmd: '/usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -' } [ ERR ] Periodic sending of process list error: { [Error: Command failed: /usr/bin/zabbix_sender --config /etc/zabbix/zabbix_agentd.conf --input-file -

6ea3fdf1-7004-438e-b883-e9f40689b4ea 38bfec84-fcc3-4bad-acf0-4a9fc2da2aaa 368df3c7-9f97-45ff-b588-936fe0e2d830 a7e62a9c-7890-4c6d-97d9-7feeced47c86 f0e7e155-3aaa-4ef5-ae5a-15d381e78846

eg8320 commented 8 years ago

52ec0120-aaf3-4aef-9909-94a6d7968340

rkaw92 commented 8 years ago

Discovery looks OK. It seems that the server is not accepting the data items. Let's try sending some fake data manually:

zabbix_sender -c /etc/zabbix/zabbix_agentd.conf -k "pm2.processes[www-0,cpu]" -s `hostname` -o 90 -vv

This simulates sending "CPU: 90%" for your www-0 process to zabbix. Let's see if the server accepts that and we'll work from there.

eg8320 commented 8 years ago

[root@upelk2 ~]# zabbix_sender -c /etc/zabbix/zabbix_agentd.conf -k "pm2.processes[www-0,cpu]" -s 10.10.12.70 -o 90 -vv
zabbix_sender [23136]: DEBUG: answer [{"response":"success","info":"processed: 0; failed: 1; total: 1; seconds spent: 0.000018"}] info from server: "processed: 0; failed: 1; total: 1; seconds spent: 0.000018" sent: 1; skipped: 0; total: 1

eg8320 commented 8 years ago

sorry, the 'hostname' is client name, the result is successed , I recevied host alert of email ,Please refer to the following results。I find host graph is OK of Manual exec above command. [root@upelk2 ~]# zabbix_sender -c /etc/zabbix/zabbix_agentd.conf -k "pm2.processes[www-0,cpu]" -s 10.10.12.77 -o 90 -vv zabbix_sender [23877]: DEBUG: answer [{"response":"success","info":"processed: 1; failed: 0; total: 1; seconds spent: 0.000192"}] info from server: "processed: 1; failed: 0; total: 1; seconds spent: 0.000192" sent: 1; skipped: 0; total: 1

rkaw92 commented 8 years ago

Maybe the hostname is the issue. In my testing, hosts could not write other hosts' data items. One thing to do would be to check what hostname is actually being sent to the server. As per /etc/zabbix/zabbix_agentd.conf:

### Option: Hostname
#       Unique, case sensitive hostname.
#       Required for active checks and must match hostname as configured on the server.
#       Value is acquired from HostnameItem if undefined.
#
# Mandatory: no
# Default:  
# Hostname=

### Option: HostnameItem
#       Item used for generating Hostname if it is undefined. Ignored if Hostname is defined.
#       Does not support UserParameters or aliases.
#
# Mandatory: no
# Default:
HostnameItem=system.hostname

In particular, note "Required for active checks and must match hostname as configured on the server". This means that if the hostname configured on the server in "Hosts" is 10.10.12.70, but your hostname yields upelk2, the names will not match and the active check may fail. This is also probably why passive checks (server → client) are OK - I think the hostname is not used for this kind of checks.

Can you configure the host on the Zabbix Server so that its hostname === hostname returned by the host that you are trying to run pm2-zabbix on? Another solution is to set Hostname= and comment out HostnameItem.

eg8320 commented 8 years ago

Thanks for your help, Sure enough, the problem is caused by a host name。I modify Zabbix Server web configure hostname and client hostname config in zabbix_agentd.conf, pm2-zabbix --monitor --debug is ok, thank you very much! [root@upelk2 public]# pm2-zabbix --monitor --debug

rkaw92 commented 8 years ago

I'll add a troubleshooting checklist with your issue as the first entry, so that others may find it and debug similar issues in the future. Glad to help.

biwoodfengs commented 6 years ago

I meet this issue too and finally I change the hostname in ../lib/node_modules/pm2-zabbix/monitor.js to my
zabbix server web's server hostname, then all works

ang3lwitch commented 6 years ago

ya, ok.

This solved my issue to with #19

as a note, the script is right in looking for hostname only.

Where it fails: If I were to change the following files to FQDN it would still fail and also in webui for hostname /etc/zabbix/zabbix_agentd.conf /usr/lib/node_modules/pm2-zabbix/monitor.js

In short, it works with the short name everywhere.

Thanks biwoodfengs

ang3lwitch commented 6 years ago

Another fix is do this: where your pm2-zabbix is installed run "npm install fqdn ." // this will install fqdn module into pm2-zabbix module directory

from there you will need to modify monitor.js

###Start### var fqdn = require('fqdn');

fqdn(function(err, res){ if(err){ throw err; }

console.log(res); }); var hostname = fqdn(); ##End###

where used to be: var hostname = os.hostname();

all two variables and hostname need to match.

rkaw92 commented 6 years ago

@ang3lwitch @biwoodfengs You can run pm2-zabbix --hostname <hostname> instead - there is no need to edit the script. In shells such as bash you should also be able to use command substitution like --hostname $(hostname --fqdn).

The root cause behind this apparent weirdness seems to be that the agent discovery script uses the agent config file, which contains some hostname configuration. Then, the key is created on the Zabbix server for the particular hostname that the agent has sent. If pm2-zabbix sends a different hostname, it must fail because the hostnames don't match and the server cannot accept the monitoring values.

One other thing to try would be calling with --hostname - - A value of - should cause the sender to use the values from the agent config, which should always match the agent's configuration (it's the same file!)/

rkaw92 commented 6 years ago

Looks like all is sorted here.