shinken-solutions / shinken

Flexible and scalable monitoring framework
http://www.shinken-monitoring.org
GNU Affero General Public License v3.0
1.13k stars 336 forks source link

broker issue #772

Closed pushou closed 9 years ago

pushou commented 11 years ago

Broker crashed after beeing launched traceback:

013-03-06 11:03:02,677 [1362564182] Debug : [Livestatus Query Metainfo] ge_contains_filters: [] 2013-03-06 11:03:02,677 [1362564182] Debug : [Livestatus Query Metainfo] unique_ge_contains_filters: [] 2013-03-06 11:03:02,678 [1362564182] Error : [Livestatus Query] Error: 'Hosts' object has no attribute 'itersorted' 2013-03-06 11:03:02,678 [1362564182] Debug : [Livestatus] Request duration 0.0028s 2013-03-06 11:03:02,678 [1362564182] Debug : [Livestatus Broker] Connection 11 is idle since 0 seconds (receiving) 2013-03-06 11:03:02,699 [1362564182] Debug : [Livestatus Broker] Connection 11 is idle since 0 seconds (receiving) 2013-03-06 11:03:02,700 [1362564182] Debug : [Livestatus Broker] Shutdown socket 11 2013-03-06 11:03:02,724 [1362564182] Debug : [broker-1] 7522 Broks get in 0.509371042252 2013-03-06 11:03:02,729 [1362564182] Debug : [broker-1] 0 Broks get in 0.000684976577759 2013-03-06 11:03:02,730 [1362564182] Debug : [broker-1] 0 Broks get in 0.000482082366943 2013-03-06 11:03:04,027 [1362564184] Debug : [broker-1] Time to send 9303 broks (1 secs) 2013-03-06 11:03:05,398 [1362564185] Critical : [broker-1] I got an unrecoverable error. I have to exit 2013-03-06 11:03:05,659 [1362564185] Critical : [broker-1] You can log a bug ticket at https://github.com/naparuba/shinken/issues/new to get help 2013-03-06 11:03:06,140 [1362564186] Critical : [broker-1] Back trace of it: Traceback (most recent call last): File "/usr/local/shinken/shinken/daemons/brokerdaemon.py", line 679, in main self.do_mainloop() File "/usr/local/shinken/shinken/daemon.py", line 244, in do_mainloop self.do_loop_turn() File "/usr/local/shinken/shinken/daemons/brokerdaemon.py", line 625, in do_loop_turn self.get_objects_from_from_queues() File "/usr/local/shinken/shinken/daemons/brokerdaemon.py", line 273, in get_objects_from_from_queues o = f.get(block=False) File "", line 2, in get File "/usr/lib/python2.6/multiprocessing/managers.py", line 725, in _callmethod conn.send((self._id, methodname, args, kwds)) IOError: [Errno 32] Broken pipe

pushou commented 11 years ago

this is linked to to the load we have the same result every time we are over 8000/100000 services ( even with a fresh install )

naparuba commented 11 years ago

What about wit ha debug launch? Do you see other errors than this one (seems to be an impact, not the root problem of this bug).

On Tue, Mar 19, 2013 at 6:28 AM, pushou notifications@github.com wrote:

this is linked to to the load we have the same result every time we are over 8000/100000 services ( even with a fresh install )

— Reply to this email directly or view it on GitHubhttps://github.com/naparuba/shinken/issues/772#issuecomment-15098204 .

elyrtan commented 11 years ago

HI, i get the same error, I only have about 1500 services though :/ any leads ?

Frescha commented 10 years ago

any updates?

Seb-Solon commented 10 years ago

Still the case? Try on upstream please

Seb-Solon commented 9 years ago

Closing this. AFAIN the itersorted in livestatus is "normal". It's a race condition at the beginning that vanish after