jasonmcintosh / rabbitmq-zabbix

Zabbix RabbitMQ Configuration
Apache License 2.0
251 stars 168 forks source link

No data on Zabbix Server, INFO: Found return code of 2 in log #98

Closed thefactor82 closed 6 years ago

thefactor82 commented 6 years ago

Hi everyone, I have a problem with this template... :-( I've installed it on a rabbit machine and imported the template on zabbix server. I had some problems with the permissions configuration of the user in rab.auth file but now seems everything ok. Zabbix server has done the discovery (I have ITEMS discovered on the host with all my queues), but no data is gathered from Zabbix server (or, as I can understand, no data is sent from the rabbit machine to the zabbix server). On Zabbix server I have the trigger alarms that says "No data received on queue XXXXXXXXXXXXXXXX in 10 minutes. On Rabbit machine, in the rabbitmq_zabbix.log I have every 30secs "INFO: Found return code of 2" If I run the command "zabbix_agentd -t rabbitmq[queues]" on the rabbit machine the result is rabbitmq[queues] [t|2]

I have installed zabbix_sender on the rabbit machine, and configured the rab auth file. Also if I run the sh scripts from rabbit machine console, I receive the output: scripts/rabbitmq/list_rabbit_nodes.sh {"data": [{"{#NODETYPE}": "disc", "{#NODENAME}": "rabbit-node1"}

What am I doing wrong? What can I check? Is there a command to test the zabbix_sender part in order to understand if it's working as it should? Could you please help me? Thanks in advance

thefactor82 commented 6 years ago

Little update. I've changed log level on rabbitmq_zabbix.log from info to debug. I can see that data is gathered from rabbit, but says that:

zabbix_sender [30646]: DEBUG: answer [{"response":"success","info":"processed: 0; failed: 135; total: 135; seconds spent: 0.002427"

Aytuar commented 6 years ago

I have the same problem. Here is the log from the server.

  1281:20180816:165633.000 trapper got '{"request":"sender data","data":[{"host":"hostname.local","key":"rabbitmq.queues[/,queue_memory,aliveness-test]","value":"22008"},
#   (many data approximately 20 KB, 119 keys)
  1281:20180816:165633.052 In recv_senderhistory()
  1281:20180816:165633.110 In process_client_history_data()
  1281:20180816:165633.168 In parse_history_data()
  1281:20180816:165633.221 End of parse_history_data():SUCCEED processed:119/119
  1281:20180816:165633.272 In process_history_data()
  1281:20180816:165633.322 End of process_history_data() processed:0
  1281:20180816:165633.375 End of process_client_history_data():SUCCEED
  1281:20180816:165633.427 In zbx_send_response()
  1281:20180816:165633.475 zbx_send_response() '{"response":"success","info":"processed: 0; failed: 119; total: 119; seconds spent: 0.211884"}'
  1281:20180816:165633.525 End of zbx_send_response():SUCCEED
  1281:20180816:165633.575 End of recv_senderhistory()
  1281:20180816:165633.674 __zbx_zbx_setproctitle() title:'trapper #79 [processed data in 0.634828 sec, waiting for connection]'
  1281:20180816:165633.728 __zbx_zbx_setproctitle() title:'trapper #79 [processing data]'
thefactor82 commented 6 years ago

I've managed to solve it! In fact... It was a problem of different hostnames between the rabbit server and the zabbix server. We have linux machines joined to domain here so in api.py the declaration

self.senderhostname = senderhostname or socket.gethostname()

gets the FQDN of my rabbit machine... But in Zabbix I've entered it with only the hostname and not the FQDN.

This kind of problem was also in other thread here, so sorry for opening a new one.

Aytuar commented 6 years ago

You're right. The problem is in the api.py script. There, as the host name for zabbix-server, the full FQDN of the hostname was sent, and on the server the host was registered with a truncated one. Solves

  1. or by fixing api.py (I did not)
  2. or the parameter HOSTNAME = "your-hostname" in .rab.auth, and also by adding the parameter in rabbitmq-status.sh for the script api.py --senderhostname = $HOSTNAME My second method works.
alsheliak commented 6 years ago

I have the same error I'm pulling my hairs trying to solve it.

2018-08-20 14:18:01,809 DEBUG: zabbix_sender [50096]: DEBUG: answer [{"response":"success","info":"processed: 0; failed: 126; total: 126; seconds spent: 0.001090"}] 2018-08-20 14:18:31,373 DEBUG: zabbix_sender [50767]: DEBUG: answer [{"response":"success","info":"processed: 0; failed: 126; total: 126; seconds spent: 0.001098"}] 2018-08-20 14:19:02,073 DEBUG: zabbix_sender [53753]: DEBUG: answer [{"response":"success","info":"processed: 0; failed: 126; total: 126; seconds spent: 0.001095"}]

jasonmcintosh commented 6 years ago

Anytime you see: [{"response":"success","info":"processed: 0; failed: 135; total: 135; seconds spent: 0.002427" That tells you it scanned 135 items, tried to send it to zabbix and zabbix couldn't find a value for them. It can be a hostname mismatch, it can also be that discovery hasn't created zabbix entries for the items. Depends entirely on the use case. @Aytuar That's why I added the HOSTNAME option to the API & script :) There are other more interesting use cases - the default zabbix_sender uses the config file on the server, but that's not always appropriate, and there are soooo many possibilities here that it's hard to cover them all.

Here's some things I've used to debug this kinda stuff: 1) Get one of the keys manually (e.g. rabbitmq[whatever]) and try doing zabbix_sender manually (read the docs, it's not impossible, just have to work out the right arguments. TYPICALLY you'll find files in /tmp for failed requests so you can see the format of what it's trying to send) 2) From the server, try calling zabbix_get to query the client for information 3) Try changing configs from active to passive as this can change configs depending on your installation.

jasonmcintosh commented 6 years ago

Closing this issue can re-open if there are still problems