Telegram Sender: Uncaught ConnectionError from requests library

[mero64]

This error crashed my service yesterday. I was on 014e987, but it looks like the code is still the same (sorry if fixed already).

18:15:08 flathunter[210512]: Traceback (most recent call last):
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
18:15:08 flathunter[210512]:     conn = connection.create_connection(
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/urllib3/util/connection.py", line 72, in create_connection
18:15:08 flathunter[210512]:     for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
18:15:08 flathunter[210512]:   File "/usr/lib/python3.9/socket.py", line 953, in getaddrinfo
18:15:08 flathunter[210512]:     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
18:15:08 flathunter[210512]: socket.gaierror: [Errno -3] Temporary failure in name resolution
18:15:08 flathunter[210512]: During handling of the above exception, another exception occurred:
18:15:08 flathunter[210512]: Traceback (most recent call last):
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
18:15:08 flathunter[210512]:     httplib_response = self._make_request(
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/urllib3/connectionpool.py", line 386, in _make_request
18:15:08 flathunter[210512]:     self._validate_conn(conn)
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
18:15:08 flathunter[210512]:     conn.connect()
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/urllib3/connection.py", line 358, in connect
18:15:08 flathunter[210512]:     self.sock = conn = self._new_conn()
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
18:15:08 flathunter[210512]:     raise NewConnectionError(
18:15:08 flathunter[210512]: urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7fe24842c7f0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution
18:15:08 flathunter[210512]: During handling of the above exception, another exception occurred:
18:15:08 flathunter[210512]: Traceback (most recent call last):
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/requests/adapters.py", line 489, in send
18:15:08 flathunter[210512]:     resp = conn.urlopen(
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
18:15:08 flathunter[210512]:     retries = retries.increment(
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment
18:15:08 flathunter[210512]:     raise MaxRetryError(_pool, url, error or ResponseError(cause))
18:15:08 flathunter[210512]: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.telegram.org', port=443): Max retries exceeded with url: /bot<censoredToken>/sendMessage (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fe24842c7f0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
18:15:08 flathunter[210512]: During handling of the above exception, another exception occurred:
18:15:08 flathunter[210512]: Traceback (most recent call last):
18:15:08 flathunter[210512]:   File "/home/flathunter/flathunter/flathunt.py", line 109, in <module>
18:15:08 flathunter[210512]:     main()
18:15:08 flathunter[210512]:   File "/home/flathunter/flathunter/flathunt.py", line 105, in main
18:15:08 flathunter[210512]:     launch_flat_hunt(config, heartbeat)
18:15:08 flathunter[210512]:   File "/home/flathunter/flathunter/flathunt.py", line 36, in launch_flat_hunt
18:15:08 flathunter[210512]:     hunter.hunt_flats()
18:15:08 flathunter[210512]:   File "/home/flathunter/flathunter/flathunter/hunter.py", line 54, in hunt_flats
18:15:08 flathunter[210512]:     for expose in processor_chain.process(self.crawl_for_exposes(max_pages)):
18:15:08 flathunter[210512]:   File "/home/flathunter/flathunter/flathunter/sender_telegram.py", line 34, in process_expose
18:15:08 flathunter[210512]:     self.__broadcast(
18:15:08 flathunter[210512]:   File "/home/flathunter/flathunter/flathunter/sender_telegram.py", line 53, in __broadcast
18:15:08 flathunter[210512]:     msg = self.__send_text(receiver, message)
18:15:08 flathunter[210512]:   File "/home/flathunter/flathunter/flathunter/sender_telegram.py", line 85, in __send_text
18:15:08 flathunter[210512]:     response = requests.request("POST", self.__text_message_url, data=payload, timeout=30)
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/requests/api.py", line 59, in request
18:15:08 flathunter[210512]:     return session.request(method=method, url=url, **kwargs)
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/requests/sessions.py", line 587, in request
18:15:08 flathunter[210512]:     resp = self.send(prep, **send_kwargs)
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/requests/sessions.py", line 701, in send
18:15:08 flathunter[210512]:     r = adapter.send(request, **kwargs)
18:15:08 flathunter[210512]:   File "/home/flathunter/.local/share/virtualenvs/flathunter-2Wmg4yEE/lib/python3.9/site-packages/requests/adapters.py", line 565, in send
18:15:08 flathunter[210512]:     raise ConnectionError(e, request=request)
18:15:08 flathunter[210512]:  requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.telegram.org', port=443): Max retries exceeded with url: /bot<censoredToken>/sendMessage (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fe24842c7f0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
18:15:08 systemd[1]: flathunter.service: Main process exited, code=exited, status=1/FAILURE
18:15:08 systemd[1]: flathunter.service: Failed with result 'exit-code'.

[codders]

Hi @mero64 ,

This looks like it might be just a temporary problem connecting to the Telegram server (in order to send the notification messages). Did the problem persist? Are you still seeing this?

[mero64]

Hi, yes, it seems to be a problem reaching the Telegram server. Since it is an external dependency, I think the error should be caught, in case the server is unreachable. Just to make the service more stable. It is very rare (but I guess that depends on Telegrams stability). I haven't seen it again.

As a side note, I got an apartment a few days ago thanks to the flathunter, so thank you!

[codders]

Hi @mero64 ,

If we catch / suppress the error, we have no way to notify the user / administrator that the notifications are not being sent. We need to be careful only to catch 500 errors and timeouts (and not 4xx errors that might indicate a configuration issue). I think because the issue is rare, and because a flathunter that doesn't send notifications is pretty useless, I would leave this as it is for now. But if you're interested to make a pull request I would welcome that :)

And congrats on finding a flat! We don't get so many user experience reports because people who have flats tend to disengage with the software :) Nice to hear that it helped you.

[codders]

Closing this - pull requests still welcome.