Closed itsx closed 4 years ago
Function setup_metrics was throwing an exception in the exception handler of _monitor method of TaskThread:
https://github.com/OvalMoney/celery-exporter/blob/497c8d8bb39d4a808805d1f3340fd84eaff6d0b1/celery_exporter/monitor.py#L64
Exception:
File "/usr/local/lib/python3.6/site-packages/celery_exporter/monitor.py", line 122, in setup_metrics for name, labels, cnt in metric.samples: ValueError: too many values to unpack (expected 3)
Problem was that metrics.samples iterable contains 5 items instead of three ('timestamp' and 'exemplar' were not anticipated): https://github.com/OvalMoney/celery-exporter/blob/497c8d8bb39d4a808805d1f3340fd84eaff6d0b1/celery_exporter/monitor.py#L122
Major problem: This exception thrown in an exception handler caused hanging of the task thread after lost of the connection to a broker. Normally exception from the lost connection is handled in an exception handler and recovery loop: https://github.com/OvalMoney/celery-exporter/blob/497c8d8bb39d4a808805d1f3340fd84eaff6d0b1/celery_exporter/monitor.py#L53 starts new connection. Because of the second exception in the handler, this has never happened.
After lost of a connection to a broker, other metrics from other threads were collected without problems.
Fixes #29
Function setup_metrics was throwing an exception in the exception handler of _monitor method of TaskThread:
https://github.com/OvalMoney/celery-exporter/blob/497c8d8bb39d4a808805d1f3340fd84eaff6d0b1/celery_exporter/monitor.py#L64
Exception:
Problem was that metrics.samples iterable contains 5 items instead of three ('timestamp' and 'exemplar' were not anticipated): https://github.com/OvalMoney/celery-exporter/blob/497c8d8bb39d4a808805d1f3340fd84eaff6d0b1/celery_exporter/monitor.py#L122
Major problem: This exception thrown in an exception handler caused hanging of the task thread after lost of the connection to a broker. Normally exception from the lost connection is handled in an exception handler and recovery loop: https://github.com/OvalMoney/celery-exporter/blob/497c8d8bb39d4a808805d1f3340fd84eaff6d0b1/celery_exporter/monitor.py#L53 starts new connection. Because of the second exception in the handler, this has never happened.
After lost of a connection to a broker, other metrics from other threads were collected without problems.
Fixes #29