search

goetzk / ansible-shinken

Ansible configuration role for Shinken monitoring system
1 stars 2 forks source link

Restart handlers are unreliable #6

Open goetzk opened 7 years ago

goetzk commented 7 years ago

The service restarts are unreliable on amazon (and possibly others)

[...]

RUNNING HANDLER [goetzk.shinken : restart shinken arbiter] *********************
fatal: [13.55.21.151]: FAILED! => {"changed": false, "failed": true, "msg": "/etc/rc.d/init.d/shinken: line 67: cd: /etc/rc.d/init.d/../../var: No such file or directory\n/etc/rc.d/init.d/shinken: line 68: cd: /etc/rc.d/init.d/../../etc: No such file or directory\n/etc/rc.d/init.d/shinken: line 67: cd: /etc/rc.d/init.d/../../var: No such file or directory\n/etc/rc.d/init.d/shinken: line 68: cd: /etc/rc.d/init.d/../../etc: No such file or directory\n"}
[...]
PLAY RECAP *********************************************************************
123.123.321.321               : ok=3    changed=1    unreachable=0    failed=1

This is using ansible so may be an upstream bug.

cat roles/goetzk.shinken/handlers/main.yml 
---
- name: restart shinken arbiter
  service: name=shinken-arbiter state=restarted

- name: restart shinken broker
  service: name=shinken-broker state=restarted
goetzk commented 7 years ago

Looks like this is specific to calling -arbiter, rather than restarting shinken itself. The bug is therefor going to be in the init script/s

 sudo /etc/init.d/shinken restart
Restarting scheduler
Restarting poller                                          [  OK  ]
Restarting reactionner                                     [  OK  ]
Restarting broker                                          [  OK  ]
Restarting receiver                                        [  OK  ]
Restarting arbiter                                         [  OK  ]
Doing config check
sudo /etc/init.d/shinken-arbiter restart
/etc/rc.d/init.d/shinken: line 67: cd: /etc/rc.d/init.d/../../var: No such file or directory
/etc/rc.d/init.d/shinken: line 68: cd: /etc/rc.d/init.d/../../etc: No such file or directory
Restarting arbiter
Doing config check
                     [  OK  ]

also happens for broker.

sudo /etc/init.d/shinken-broker restart
/etc/rc.d/init.d/shinken: line 67: cd: /etc/rc.d/init.d/../../var: No such file or directory
/etc/rc.d/init.d/shinken: line 68: cd: /etc/rc.d/init.d/../../etc: No such file or directory
Restarting broker
                         [  OK  ]
goetzk commented 7 years ago

I've just opened https://github.com/naparuba/shinken/issues/1901 upstream about this.