search

qk4l / zabbix-cachet

Python script which sync Zabbix IT Services with Cachet
MIT License
80 stars 32 forks source link

Not syncing properly #34

Closed DonRichards closed 6 years ago

DonRichards commented 6 years ago

Issue is when there is an issue on Zabbix the issue isn't showing up on CachetHQ. How do I correct this? I've checked that the service/script is running. I've tried restarting it. I've also tried to manually set it to each of the settings and then back and it still is reporting that there is no issue.

Zabbix is reporting an issue.

0

The Frontend dashboard

screenshot-status lib utk edu-2018 04 24-10-26-37

Expanded the last one that is experiencing issues.

screenshot-status lib utk edu-2018 04 24-11-08-33

qk4l commented 6 years ago

Hi, Have you checked debug log output?

DonRichards commented 6 years ago

Which ones?

DonRichards commented 6 years ago

I should have said that I don't follow. I don't see where this creates a log itself. Can you be specific?

qk4l commented 6 years ago

I meant zabbix-cachet logs.

Depend on your installation they could be saved in different places.

The most simple way to find them is to run script from console, logs will be in stdout.

Also I recommend to set log level to debug mode (config.yml file) if logs will not show a problem.

log_level: DEBUG

If you provide debug logs I could check them...

DonRichards commented 6 years ago

Sorry, just noticed. I'll add that and get back.

DonRichards commented 6 years ago

OK here we go. It's failing. 

Apr 11 06:00:51 cachethq_server systemd[1]: Started Zabbix to Cachet Sync Service.
Apr 11 06:00:51 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service...
Apr 11 06:00:52 cachethq_server zabbix-cachet.py[747]: 2018-04-11 06:00:52 EDT INFO: (MainThread) Zabbix Cachet v.1.3.4 started
Apr 11 06:00:52 cachethq_server zabbix-cachet.py[747]: 2018-04-11 06:00:52 EDT INFO: (MainThread) JSON-RPC Server Endpoint: https://zabbix.utk.edu/api_jsonrpc.php
Apr 11 06:00:52 cachethq_server zabbix-cachet.py[747]: 2018-04-11 06:00:52 EDT ERROR: (MainThread) HTTPSConnectionPool(host='zabbix.utk.edu', port=443): Max retries exceeded with ur
Apr 11 06:00:52 cachethq_server systemd[1]: zabbix_to_cachethq.service: main process exited, code=exited, status=1/FAILURE
Apr 11 06:00:52 cachethq_server systemd[1]: Unit zabbix_to_cachethq.service entered failed state.
Apr 11 06:00:52 cachethq_server systemd[1]: zabbix_to_cachethq.service failed.
Apr 18 11:43:09 cachethq_server systemd[1]: Started Zabbix to Cachet Sync Service.
Apr 18 11:43:09 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service...
Apr 18 11:43:10 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:10 EDT INFO: (MainThread) Zabbix Cachet v.1.3.4 started
Apr 18 11:43:10 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:10 EDT INFO: (MainThread) JSON-RPC Server Endpoint: https://zabbix.utk.edu/api_jsonrpc.php
Apr 18 11:43:10 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:10 EDT INFO: (MainThread) Zabbix ver: 3.2.11. Cachet ver: 2.3.14-dev
Apr 18 11:43:10 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:10 EDT INFO: (MainThread) Component 7th Check was created in group id 7.
Apr 18 11:43:11 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:11 EDT INFO: (MainThread) Component 6th Check was created in group id 7.
Apr 18 11:43:11 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:11 EDT INFO: (MainThread) Component 4th Check was created in group id 7.
Apr 18 11:43:11 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:11 EDT INFO: (MainThread) Component 1st Check was created in group id 7.
Apr 18 11:43:11 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:11 EDT INFO: (MainThread) Component 2nd Check was created in group id 7.
Apr 18 11:43:16 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:16 EDT INFO: (MainThread) Component 5th Check was created in group id 7.
Apr 18 11:43:17 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:17 EDT INFO: (MainThread) Component 3rd Check was created in group id 7.
Apr 18 11:43:17 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:17 EDT INFO: (MainThread) Component Libcal Ping rate was created in group id 8.
Apr 18 11:43:17 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:17 EDT INFO: (MainThread) Component Libcal ICMP down was created in group id 8.
Apr 18 11:43:18 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:18 EDT INFO: (MainThread) Component Group Digital Collections (Islandora) was created (9)
Apr 18 11:43:18 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:18 EDT INFO: (MainThread) Component Digital Zabbix agent was created in group id 9.
Apr 18 11:43:18 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:18 EDT INFO: (MainThread) Component Digital HTTPS service was created in group id 9.
Apr 18 11:43:18 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:18 EDT INFO: (MainThread) Component Digital HTTP service was created in group id 9.
Apr 18 11:43:18 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:18 EDT INFO: (MainThread) Component Digital DB was created in group id 9.
Apr 18 11:43:19 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:19 EDT INFO: (MainThread) Component Group Main site (www.lib.utk.edu) was created (10)
Apr 18 11:43:19 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:19 EDT INFO: (MainThread) Component www Zabbix was created in group id 10.
Apr 18 11:43:19 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:19 EDT INFO: (MainThread) Component www db was created in group id 10.
Apr 18 11:43:19 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:19 EDT INFO: (MainThread) Component www apache was created in group id 10.
Apr 18 11:43:20 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:20 EDT INFO: (MainThread) Component www https was created in group id 10.
Apr 18 11:43:20 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:20 EDT INFO: (MainThread) Component www http was created in group id 10.
Apr 18 11:43:20 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:20 EDT INFO: (MainThread) Successfully synced Cachet components with Zabbix Services
Apr 18 11:43:20 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:20 EDT INFO: (MainThread) Restart triggers_watcher worker
Apr 18 11:43:20 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:20 EDT INFO: (Trigger Watcher) start trigger watcher
Apr 18 12:10:15 cachethq_server systemd[1]: Stopping Zabbix to Cachet Sync Service...
Apr 18 12:10:15 cachethq_server systemd[1]: Started Zabbix to Cachet Sync Service.
Apr 18 12:10:15 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service...
Apr 18 12:10:15 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:15 EDT INFO: (MainThread) Zabbix Cachet v.1.3.4 started
Apr 18 12:10:15 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:15 EDT INFO: (MainThread) JSON-RPC Server Endpoint: https://zabbix.utk.edu/api_jsonrpc.php
Apr 18 12:10:16 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:16 EDT INFO: (MainThread) Zabbix ver: 3.2.11. Cachet ver: 2.3.14-dev
Apr 18 12:10:20 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:20 EDT INFO: (MainThread) Successfully synced Cachet components with Zabbix Services
Apr 18 12:10:20 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:20 EDT INFO: (MainThread) Restart triggers_watcher worker
Apr 18 12:10:20 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:20 EDT INFO: (Trigger Watcher) start trigger watcher
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: Exception in thread Trigger Watcher:
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: Traceback (most recent call last):
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: File "/usr/lib64/python3.4/threading.py", line 911, in _bootstrap_inner
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: self.run()
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: File "/usr/lib64/python3.4/threading.py", line 859, in run
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: self._target(*self._args, **self._kwargs)
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: File "/home/don/zabbix-cachet/zabbix-cachet.py", line 595, in triggers_watcher_worker
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: triggers_watcher(service_map)
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: File "/home/don/zabbix-cachet/zabbix-cachet.py", line 520, in triggers_watcher
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: if zbx_event['acknowledged'] == '1':
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: TypeError: list indices must be integers, not str
Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 13:10:25 EDT INFO: (MainThread) Successfully synced Cachet components with Zabbix Services
Apr 18 13:44:08 cachethq_server systemd[1]: Stopping Zabbix to Cachet Sync Service...
Apr 18 13:44:08 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service...
Apr 18 13:44:08 cachethq_server systemd[1]: zabbix_to_cachethq.service: control process exited, code=exited status=203
Apr 18 13:44:08 cachethq_server systemd[1]: Failed to start Zabbix to Cachet Sync Service.
Apr 18 13:44:08 cachethq_server systemd[1]: Unit zabbix_to_cachethq.service entered failed state.
Apr 18 13:44:08 cachethq_server systemd[1]: zabbix_to_cachethq.service failed.
Apr 18 13:44:13 cachethq_server systemd[1]: zabbix_to_cachethq.service holdoff time over, scheduling restart.
Apr 18 13:44:13 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service...
Apr 18 13:44:13 cachethq_server systemd[1]: zabbix_to_cachethq.service: control process exited, code=exited status=203
Apr 18 13:44:13 cachethq_server systemd[1]: Failed to start Zabbix to Cachet Sync Service.
Apr 18 13:44:13 cachethq_server systemd[1]: Unit zabbix_to_cachethq.service entered failed state.
Apr 18 13:44:13 cachethq_server systemd[1]: zabbix_to_cachethq.service failed.
Apr 18 13:44:18 cachethq_server systemd[1]: zabbix_to_cachethq.service holdoff time over, scheduling restart.
Apr 18 13:44:18 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service...
Apr 18 13:44:18 cachethq_server systemd[1]: zabbix_to_cachethq.service: control process exited, code=exited status=203
Apr 18 13:44:18 cachethq_server systemd[1]: Failed to start Zabbix to Cachet Sync Service.
Apr 18 13:44:18 cachethq_server systemd[1]: Unit zabbix_to_cachethq.service entered failed state.
Apr 18 13:44:18 cachethq_server systemd[1]: zabbix_to_cachethq.service failed.
Apr 18 13:44:23 cachethq_server systemd[1]: zabbix_to_cachethq.service holdoff time over, scheduling restart.
Apr 18 13:44:23 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service...
Apr 18 13:44:23 cachethq_server systemd[1]: zabbix_to_cachethq.service: control process exited, code=exited status=203
Apr 18 13:44:23 cachethq_server systemd[1]: Failed to start Zabbix to Cachet Sync Service.
Apr 18 13:44:23 cachethq_server systemd[1]: Unit zabbix_to_cachethq.service entered failed state.
Apr 18 13:44:23 cachethq_server systemd[1]: zabbix_to_cachethq.service failed.
Apr 18 13:44:29 cachethq_server systemd[1]: zabbix_to_cachethq.service holdoff time over, scheduling restart.
Apr 18 13:44:29 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service...
qk4l commented 6 years ago

Don, have you tried to read that logs?

If they are not enough informative increase log level and try again.

If you have a particular question or improve request I will be happy to help you.

But I do not provide support for such issues.

Thank you for understanding.

On Tue, May 8, 2018, 7:51 PM Don Richards notifications@github.com wrote:

OK here we go. It's failing.

Apr 11 06:00:51 cachethq_server systemd[1]: Started Zabbix to Cachet Sync Service. Apr 11 06:00:51 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service... Apr 11 06:00:52 cachethq_server zabbix-cachet.py[747]: 2018-04-11 06:00:52 EDT INFO: (MainThread) Zabbix Cachet v.1.3.4 started Apr 11 06:00:52 cachethq_server zabbix-cachet.py[747]: 2018-04-11 06:00:52 EDT INFO: (MainThread) JSON-RPC Server Endpoint: https://zabbix.utk.edu/api_jsonrpc.php Apr 11 06:00:52 cachethq_server zabbix-cachet.py[747]: 2018-04-11 06:00:52 EDT ERROR: (MainThread) HTTPSConnectionPool(host='zabbix.utk.edu', port=443): Max retries exceeded with ur Apr 11 06:00:52 cachethq_server systemd[1]: zabbix_to_cachethq.service: main process exited, code=exited, status=1/FAILURE Apr 11 06:00:52 cachethq_server systemd[1]: Unit zabbix_to_cachethq.service entered failed state. Apr 11 06:00:52 cachethq_server systemd[1]: zabbix_to_cachethq.service failed. Apr 18 11:43:09 cachethq_server systemd[1]: Started Zabbix to Cachet Sync Service. Apr 18 11:43:09 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service... Apr 18 11:43:10 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:10 EDT INFO: (MainThread) Zabbix Cachet v.1.3.4 started Apr 18 11:43:10 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:10 EDT INFO: (MainThread) JSON-RPC Server Endpoint: https://zabbix.utk.edu/api_jsonrpc.php Apr 18 11:43:10 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:10 EDT INFO: (MainThread) Zabbix ver: 3.2.11. Cachet ver: 2.3.14-dev Apr 18 11:43:10 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:10 EDT INFO: (MainThread) Component 7th Check was created in group id 7. Apr 18 11:43:11 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:11 EDT INFO: (MainThread) Component 6th Check was created in group id 7. Apr 18 11:43:11 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:11 EDT INFO: (MainThread) Component 4th Check was created in group id 7. Apr 18 11:43:11 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:11 EDT INFO: (MainThread) Component 1st Check was created in group id 7. Apr 18 11:43:11 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:11 EDT INFO: (MainThread) Component 2nd Check was created in group id 7. Apr 18 11:43:16 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:16 EDT INFO: (MainThread) Component 5th Check was created in group id 7. Apr 18 11:43:17 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:17 EDT INFO: (MainThread) Component 3rd Check was created in group id 7. Apr 18 11:43:17 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:17 EDT INFO: (MainThread) Component Libcal Ping rate was created in group id 8. Apr 18 11:43:17 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:17 EDT INFO: (MainThread) Component Libcal ICMP down was created in group id 8. Apr 18 11:43:18 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:18 EDT INFO: (MainThread) Component Group Digital Collections (Islandora) was created (9) Apr 18 11:43:18 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:18 EDT INFO: (MainThread) Component Digital Zabbix agent was created in group id 9. Apr 18 11:43:18 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:18 EDT INFO: (MainThread) Component Digital HTTPS service was created in group id 9. Apr 18 11:43:18 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:18 EDT INFO: (MainThread) Component Digital HTTP service was created in group id 9. Apr 18 11:43:18 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:18 EDT INFO: (MainThread) Component Digital DB was created in group id 9. Apr 18 11:43:19 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:19 EDT INFO: (MainThread) Component Group Main site (www.lib.utk.edu) was created (10) Apr 18 11:43:19 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:19 EDT INFO: (MainThread) Component www Zabbix was created in group id 10. Apr 18 11:43:19 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:19 EDT INFO: (MainThread) Component www db was created in group id 10. Apr 18 11:43:19 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:19 EDT INFO: (MainThread) Component www apache was created in group id 10. Apr 18 11:43:20 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:20 EDT INFO: (MainThread) Component www https was created in group id 10. Apr 18 11:43:20 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:20 EDT INFO: (MainThread) Component www http was created in group id 10. Apr 18 11:43:20 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:20 EDT INFO: (MainThread) Successfully synced Cachet components with Zabbix Services Apr 18 11:43:20 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:20 EDT INFO: (MainThread) Restart triggers_watcher worker Apr 18 11:43:20 cachethq_server zabbix-cachet.py[21473]: 2018-04-18 11:43:20 EDT INFO: (Trigger Watcher) start trigger watcher Apr 18 12:10:15 cachethq_server systemd[1]: Stopping Zabbix to Cachet Sync Service... Apr 18 12:10:15 cachethq_server systemd[1]: Started Zabbix to Cachet Sync Service. Apr 18 12:10:15 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service... Apr 18 12:10:15 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:15 EDT INFO: (MainThread) Zabbix Cachet v.1.3.4 started Apr 18 12:10:15 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:15 EDT INFO: (MainThread) JSON-RPC Server Endpoint: https://zabbix.utk.edu/api_jsonrpc.php Apr 18 12:10:16 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:16 EDT INFO: (MainThread) Zabbix ver: 3.2.11. Cachet ver: 2.3.14-dev Apr 18 12:10:20 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:20 EDT INFO: (MainThread) Successfully synced Cachet components with Zabbix Services Apr 18 12:10:20 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:20 EDT INFO: (MainThread) Restart triggers_watcher worker Apr 18 12:10:20 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 12:10:20 EDT INFO: (Trigger Watcher) start trigger watcher Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: Exception in thread Trigger Watcher: Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: Traceback (most recent call last): Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: File "/usr/lib64/python3.4/threading.py", line 911, in _bootstrap_inner Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: self.run() Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: File "/usr/lib64/python3.4/threading.py", line 859, in run Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: self._target(*self._args, **self._kwargs) Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: File "/home/don/zabbix-cachet/zabbix-cachet.py", line 595, in triggers_watcher_worker Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: triggers_watcher(service_map) Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: File "/home/don/zabbix-cachet/zabbix-cachet.py", line 520, in triggers_watcher Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: if zbx_event['acknowledged'] == '1': Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: TypeError: list indices must be integers, not str Apr 18 13:10:25 cachethq_server zabbix-cachet.py[21589]: 2018-04-18 13:10:25 EDT INFO: (MainThread) Successfully synced Cachet components with Zabbix Services Apr 18 13:44:08 cachethq_server systemd[1]: Stopping Zabbix to Cachet Sync Service... Apr 18 13:44:08 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service... Apr 18 13:44:08 cachethq_server systemd[1]: zabbix_to_cachethq.service: control process exited, code=exited status=203 Apr 18 13:44:08 cachethq_server systemd[1]: Failed to start Zabbix to Cachet Sync Service. Apr 18 13:44:08 cachethq_server systemd[1]: Unit zabbix_to_cachethq.service entered failed state. Apr 18 13:44:08 cachethq_server systemd[1]: zabbix_to_cachethq.service failed. Apr 18 13:44:13 cachethq_server systemd[1]: zabbix_to_cachethq.service holdoff time over, scheduling restart. Apr 18 13:44:13 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service... Apr 18 13:44:13 cachethq_server systemd[1]: zabbix_to_cachethq.service: control process exited, code=exited status=203 Apr 18 13:44:13 cachethq_server systemd[1]: Failed to start Zabbix to Cachet Sync Service. Apr 18 13:44:13 cachethq_server systemd[1]: Unit zabbix_to_cachethq.service entered failed state. Apr 18 13:44:13 cachethq_server systemd[1]: zabbix_to_cachethq.service failed. Apr 18 13:44:18 cachethq_server systemd[1]: zabbix_to_cachethq.service holdoff time over, scheduling restart. Apr 18 13:44:18 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service... Apr 18 13:44:18 cachethq_server systemd[1]: zabbix_to_cachethq.service: control process exited, code=exited status=203 Apr 18 13:44:18 cachethq_server systemd[1]: Failed to start Zabbix to Cachet Sync Service. Apr 18 13:44:18 cachethq_server systemd[1]: Unit zabbix_to_cachethq.service entered failed state. Apr 18 13:44:18 cachethq_server systemd[1]: zabbix_to_cachethq.service failed. Apr 18 13:44:23 cachethq_server systemd[1]: zabbix_to_cachethq.service holdoff time over, scheduling restart. Apr 18 13:44:23 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service... Apr 18 13:44:23 cachethq_server systemd[1]: zabbix_to_cachethq.service: control process exited, code=exited status=203 Apr 18 13:44:23 cachethq_server systemd[1]: Failed to start Zabbix to Cachet Sync Service. Apr 18 13:44:23 cachethq_server systemd[1]: Unit zabbix_to_cachethq.service entered failed state. Apr 18 13:44:23 cachethq_server systemd[1]: zabbix_to_cachethq.service failed. Apr 18 13:44:29 cachethq_server systemd[1]: zabbix_to_cachethq.service holdoff time over, scheduling restart. Apr 18 13:44:29 cachethq_server systemd[1]: Starting Zabbix to Cachet Sync Service...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/qk4l/zabbix-cachet/issues/34#issuecomment-387468790, or mute the thread https://github.com/notifications/unsubscribe-auth/AIAVSx3cVBzRO3vssuOZMlZEbhA1ezkQks5twc0ggaJpZM4TkBCY .

-- Best regards, Artyom

DonRichards commented 6 years ago

@qk4l Do have an example of a .service file for this? Maybe there is some variable or parameter that needs to be added.

DonRichards commented 6 years ago

So I think this will be a little more helpful. If I run this script manually this is what I get. 

$ ./zabbix-cachet.py
2018-05-09 16:19:23 EDT INFO: (MainThread) Zabbix Cachet v.1.3.4 started
2018-05-09 16:19:23 EDT INFO: (MainThread) JSON-RPC Server Endpoint: https://zmonitor.lib.utk.edu/api_jsonrpc.php
2018-05-09 16:19:23 EDT INFO: (MainThread) Zabbix ver: 3.2.11. Cachet ver: 2.3.14-dev
2018-05-09 16:19:28 EDT INFO: (MainThread) Successfully synced Cachet components with Zabbix Services
2018-05-09 16:19:28 EDT INFO: (MainThread) Restart triggers_watcher worker
2018-05-09 16:19:28 EDT INFO: (Trigger Watcher) start trigger watcher
Exception in thread Trigger Watcher:
Traceback (most recent call last):
  File "/usr/lib64/python3.4/threading.py", line 911, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.4/threading.py", line 859, in run
    self._target(*self._args, **self._kwargs)
  File "./zabbix-cachet.py", line 595, in triggers_watcher_worker
    triggers_watcher(service_map)
  File "./zabbix-cachet.py", line 520, in triggers_watcher
    if zbx_event['acknowledged'] == '1':
TypeError: list indices must be integers, not str
qk4l commented 6 years ago

Yes! This now clear why Trigger Watcher die but unfortunately I can't image what Zabbix`s data lead to the problem.

Could you please set log_level to debug and post output as a file.

log_level: DEBUG

Thank you.

ikonovalenko commented 6 years ago

Hello,

I have exactly the same problem. Zabbix-cachet does not post anything. I enabled DEBUG log level, however, do not see any errors.

DonRichards commented 6 years ago

With my config file roughly like this. 

zabbix:
  user: username
  pass: password
  server: https://zabbix.utk.edu
  https-verify: true

cachet:
  token: qwertyuiop1234567890
  server: https://status.utk.edu
  https-verify: true

settings:
  root_service: ''
  update_inc_interval: 120
  update_comp_interval: 3600
  log_level: DEBUG
  log_level_requests: DEBUG
templates:
  acknowledgement: "{message}\n\n###### {ack_time} by {author}\n\n______\n"
  investigating: "{group} | {component} check **failed** - {time}\n\n```{trigger_name}```"
  resolving: "__Resolved__ - {time}\n\n______\n"

Running it directly with pipenv I get the following errors

error.log

DonRichards commented 6 years ago

Sorry @qk4l it took so long to respond. I completely missed the announcement for this.

qk4l commented 6 years ago

Here, at the end of file, it asked Zabbix about event and get empty result.

Is you zabbix user that you use have enough permission to read that event info?

Is follow link show events when you are login under same user https://youzabbix.url/events.php?triggerid=13944 

2018-05-31 08:03:41 EDT DEBUG: (Trigger Watcher) Sending: {
    "method": "event.get",
    "id": 39,
    "auth": "XXXXXXXXXX",
    "jsonrpc": "2.0",
    "params": {
        "expandDescription": "true",
        "select_acknowledges": "extend",
        "value": 1,
        "object": 0,
        "objectids": "13944"
    }
}
2018-05-31 08:03:41 EDT DEBUG: (Trigger Watcher) "POST /api_jsonrpc.php HTTP/1.1" 200 37
2018-05-31 08:03:41 EDT DEBUG: (Trigger Watcher) Response Code: 200
2018-05-31 08:03:41 EDT DEBUG: (Trigger Watcher) Response Body: {
    "result": [],
    "jsonrpc": "2.0",
    "id": 39
}
qk4l commented 6 years ago

I updated zabbix-cachet so it should not fail it such cases now.

qk4l commented 6 years ago

@DonRichards Hi, have your check user permission in Zabbix? Have you tried last update?

DonRichards commented 6 years ago

OK, Worked out all of the issues. Got it running as a service without any obvious failure messages but I still am not getting the status change I'm expecting. I can see the component at https://status..utk.edu/api/v1/components?page=1 but here is the response json Looking at "SHOULD_be_FAILING_SERVER ICMP down" is reporting 0.0000 / 99.9000 on zabbix but cachethq has a green light with a value of "status": 1,

https://gist.github.com/DonRichards/eb8cae447b434aabf96462379e071e0d

qk4l commented 6 years ago

Hi,

As I wrote zabbix is not returning events for trigger 13944.

I guess that this event was deleted by housekeeper or other cleaner that you use.

            "lastchange": "1459439404", # It is 03/31/2016
            "recovery_mode": "0",
            "error": "",
            "templateid": "0",
            "expression": "{13551}=0",
            "triggerid": "13944",
            "description": "libcal is unavailable by ICMP",

Please check if trigger`s problem event is available in Zabbix. If it's not available, I recommend you to reset this trigger.

DonRichards commented 6 years ago

Ok, I'll check that out and let you know. Thanks for all of the help.