search

shinken-monitoring / mod-livestatus

Shinken module for presenting data with a MK/Livestatus comptabile interface
GNU Affero General Public License v3.0
15 stars 20 forks source link

Thruk SLA Report - mod-livestatus error with big hostgroups/servicegroups #73

Open diogouchoas opened 8 years ago

diogouchoas commented 8 years ago

I have a group with over 1000 hosts and when I try to generate an Availability Report on Thruk using Livestatus, I get this error.

[1474757832] ERROR: [broker-master] [Livestatus] Back trace of this exception: Traceback (most recent call last):
  File "/var/lib/shinken/modules/livestatus/livestatus_obj.py", line 74, in handle_request
    return self.handle_request_and_fail(data)
  File "/var/lib/shinken/modules/livestatus/livestatus_obj.py", line 135, in handle_request_and_fail
    output, keepalive = query.process_query()
  File "/var/lib/shinken/modules/livestatus/livestatus_query.py", line 283, in process_query
    return self.response.respond()
  File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 147, in respond
    responselength = 1 + self.get_response_len() # 1 for the final '\n'
  File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 142, in get_response_len
    if isinstance(rsp, LiveStatusListResponse)
  File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 83, in total_len
    for generated_data in value:
  File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 278, in make_live_data_generator
    for value in self.make_live_data_generator2(result, columns, aliases):
  File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 224, in make_live_data_generator2
    item = next(result)
  File "/var/lib/shinken/modules/livestatus/livestatus_query.py", line 612, in _get_live_data_log
    for x in self.db.get_live_data_log():
  File "/var/lib/shinken/modules/logstore-sqlite/module.py", line 560, in get_live_data_log
    for rows in rows_gen:
  File "/var/lib/shinken/modules/logstore-sqlite/module.py", line 394, in select
    self.dbcursor.execute(cmd, values)
OperationalError: Expression tree is too large (maximum depth 1000)
olivierHa commented 8 years ago

It seems to be a limitation with sqlite :

http://www.sqlite.org/limits.html

"""The SQLITE_MAX_EXPR_DEPTH parameter determines the maximum expression tree depth. If the value is 0, then no limit is enforced. The current implementation has a default value of 1000""" A quick workaround would be to use the mongodb_logstore.

An other way should be to optimise the query if possible.

Another "quick" way to fix it should be to recompile your sqlite library with a value of "0"

Olivier

2016-09-25 1:00 GMT+02:00 Diogo Uchoas notifications@github.com:

I have a group with over 1000 hosts and when I try to generate an Availability Report on Thruk using Livestatus, I get this error.

[1474757832] ERROR: [broker-master] [Livestatus] Back trace of this exception: Traceback (most recent call last): File "/var/lib/shinken/modules/livestatus/livestatus_obj.py", line 74, in handle_request return self.handle_request_and_fail(data) File "/var/lib/shinken/modules/livestatus/livestatus_obj.py", line 135, in handle_request_and_fail output, keepalive = query.process_query() File "/var/lib/shinken/modules/livestatus/livestatus_query.py", line 283, in process_query return self.response.respond() File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 147, in respond responselength = 1 + self.get_response_len() # 1 for the final '\n' File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 142, in get_response_len if isinstance(rsp, LiveStatusListResponse) File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 83, in total_len for generated_data in value: File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 278, in make_live_data_generator for value in self.make_live_data_generator2(result, columns, aliases): File "/var/lib/shinken/modules/livestatus/livestatus_response.py", line 224, in make_live_data_generator2 item = next(result) File "/var/lib/shinken/modules/livestatus/livestatus_query.py", line 612, in _get_live_data_log for x in self.db.get_live_data_log(): File "/var/lib/shinken/modules/logstore-sqlite/module.py", line 560, in get_live_data_log for rows in rows_gen: File "/var/lib/shinken/modules/logstore-sqlite/module.py", line 394, in select self.dbcursor.execute(cmd, values) OperationalError: Expression tree is too large (maximum depth 1000)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/shinken-monitoring/mod-livestatus/issues/73, or mute the thread https://github.com/notifications/unsubscribe-auth/AAxIsjNLmHYRpP63X25CjR9R47ePABtAks5qtat-gaJpZM4KFxM1 .

diogouchoas commented 8 years ago

Good to know, you think it is possible to convert the sqlite data to mongodb for use with logstore_mongodb?