Logs:
[2020-04-27 16:31:20,179] task-thread:ERROR: Connection failed Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/celery_exporter/monitor.py", line 61, in _monitor recv.capture(limit=None, timeout=None, wakeup=True) File "/usr/local/lib/python3.6/site-packages/celery/events/receiver.py", line 92, in capture return list(self.consume(limit=limit, timeout=timeout, wakeup=wakeup)) File "/usr/local/lib/python3.6/site-packages/kombu/mixins.py", line 192, in consume conn.drain_events(timeout=safety_interval) File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 315, in drain_events return self.transport.drain_events(self.connection, **kwargs) File "/usr/local/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 103, in drain_events return connection.drain_events(**kwargs) File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 500, in drain_events while not self.blocking_read(timeout): File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 505, in blocking_read frame = self.transport.read_frame() File "/usr/local/lib/python3.6/site-packages/amqp/transport.py", line 256, in read_frame frame_header = read(7, True) File "/usr/local/lib/python3.6/site-packages/amqp/transport.py", line 440, in _read s = recv(n - len(rbuf)) ConnectionResetError: [Errno 104] Connection reset by peer Exception in thread Thread-2: Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/celery_exporter/monitor.py", line 61, in _monitor recv.capture(limit=None, timeout=None, wakeup=True) File "/usr/local/lib/python3.6/site-packages/celery/events/receiver.py", line 92, in capture return list(self.consume(limit=limit, timeout=timeout, wakeup=wakeup)) File "/usr/local/lib/python3.6/site-packages/kombu/mixins.py", line 192, in consume conn.drain_events(timeout=safety_interval) File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 315, in drain_events return self.transport.drain_events(self.connection, **kwargs) File "/usr/local/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 103, in drain_events return connection.drain_events(**kwargs) File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 500, in drain_events while not self.blocking_read(timeout): File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 505, in blocking_read frame = self.transport.read_frame() File "/usr/local/lib/python3.6/site-packages/amqp/transport.py", line 256, in read_frame frame_header = read(7, True) File "/usr/local/lib/python3.6/site-packages/amqp/transport.py", line 440, in _read s = recv(n - len(rbuf)) ConnectionResetError: [Errno 104] Connection reset by peer
I am facing the following issue with my connection to rabbitmq, the connection is stable at works at the beginning but after that it gets reset.
I believe this has to do with heartbeat pings not getting back to rabbitmq and it cutting the connection.
Similar issue: https://github.com/Polyconseil/aioamqp/issues/96
Logs:
[2020-04-27 16:31:20,179] task-thread:ERROR: Connection failed Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/celery_exporter/monitor.py", line 61, in _monitor recv.capture(limit=None, timeout=None, wakeup=True) File "/usr/local/lib/python3.6/site-packages/celery/events/receiver.py", line 92, in capture return list(self.consume(limit=limit, timeout=timeout, wakeup=wakeup)) File "/usr/local/lib/python3.6/site-packages/kombu/mixins.py", line 192, in consume conn.drain_events(timeout=safety_interval) File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 315, in drain_events return self.transport.drain_events(self.connection, **kwargs) File "/usr/local/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 103, in drain_events return connection.drain_events(**kwargs) File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 500, in drain_events while not self.blocking_read(timeout): File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 505, in blocking_read frame = self.transport.read_frame() File "/usr/local/lib/python3.6/site-packages/amqp/transport.py", line 256, in read_frame frame_header = read(7, True) File "/usr/local/lib/python3.6/site-packages/amqp/transport.py", line 440, in _read s = recv(n - len(rbuf)) ConnectionResetError: [Errno 104] Connection reset by peer Exception in thread Thread-2: Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/celery_exporter/monitor.py", line 61, in _monitor recv.capture(limit=None, timeout=None, wakeup=True) File "/usr/local/lib/python3.6/site-packages/celery/events/receiver.py", line 92, in capture return list(self.consume(limit=limit, timeout=timeout, wakeup=wakeup)) File "/usr/local/lib/python3.6/site-packages/kombu/mixins.py", line 192, in consume conn.drain_events(timeout=safety_interval) File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 315, in drain_events return self.transport.drain_events(self.connection, **kwargs) File "/usr/local/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 103, in drain_events return connection.drain_events(**kwargs) File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 500, in drain_events while not self.blocking_read(timeout): File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 505, in blocking_read frame = self.transport.read_frame() File "/usr/local/lib/python3.6/site-packages/amqp/transport.py", line 256, in read_frame frame_header = read(7, True) File "/usr/local/lib/python3.6/site-packages/amqp/transport.py", line 440, in _read s = recv(n - len(rbuf)) ConnectionResetError: [Errno 104] Connection reset by peer