Enable optimization for Prometheus Exporter when recording is enabled

omidmaldar commented 2 years ago

What

With the current setup RECORDINGS_METRICS_READ_FROM_DISK is always set to false. This causes the api calls for collecting ecordings metrics lasts over a few minutes if there are too many recoordings available. Changes proposed in this pr lets user to decied if they want to change it to true and benefit from faster (less than a second) metric collection.

Why

Promethues exporter spends substantial time for collecting recording metrics via bbb api. The operation takes very long time especially when there are many recordings available. The consequence of api call could be a timeout error which is logged in the docker container as follows. As a result of the error, Grafana encounters lack of data too.

Example of error message

2021-10-09 09:47:27,016 [INFO]: Collecting metrics from BigBlueButton API
2021-10-09 09:47:27,801 [INFO]: Finished collecting metrics from BigBlueButton API
----------------------------------------
Exception happened during processing of request from ('10.7.7.1', 33354)
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/socketserver.py", line 650, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/local/lib/python3.8/socketserver.py", line 360, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/local/lib/python3.8/socketserver.py", line 720, in __init__
    self.handle()
  File "/usr/local/lib/python3.8/http/server.py", line 427, in handle
    self.handle_one_request()
  File "/usr/local/lib/python3.8/http/server.py", line 415, in handle_one_request
    method()
  File "/usr/local/lib/python3.8/site-packages/prometheus_client/exposition.py", line 159, in do_GET
    self.wfile.write(output)
  File "/usr/local/lib/python3.8/socketserver.py", line 799, in write
    self._sock.sendall(b)
BrokenPipeError: [Errno 32] Broken pipe
----------------------------------------
2021-10-09 09:47:41,215 [INFO]: Finished collecting metrics from BigBlueButton API
2021-10-09 09:47:42,016 [INFO]: Collecting metrics from BigBlueButton API

Example of lack of data on Grafana panel

Screenshot from 2021-10-17 01-10-44

The error may also implies that the bbb server is down/offline.

References

alangecker commented 2 years ago

again a nicely written PR! :)

just found a minor thing with a change proposal you could accept and there are apparently some merge conflicts. Can you imagine resolving them? otherwise I could also have a look, but it seems like I don't have write access to that branch, so I would need to fork that and create a new PR... 🙃

omidmaldar commented 2 years ago

Hi there Thanks for your comment. Sounds reasonable. I'll update the pr and resolve the conflict. Good timing 🙂

bigbluebutton / docker