Multiple exceptions on connection failure

[samuel-jimenez]

When a connection fails, the following appears in the logs for each failed item.

Could this be made less spammy?

Operating System: Arch Linux Kernel Version: 6.8.4-zen1-1-zen (64-bit) python 3.11.8 urllib3 1.26.18

ERROR: HTTPConnectionPool(host='mirror.sentries.org', port=80): Max retries exceeded with url: /sanesecurity/badmacro.ndb.sha256 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x745cb54d2fd0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/urllib3/util/connection.py", line 72, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 962, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.11/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/lib/python3.11/http/client.py", line 1298, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1344, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1293, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.11/http/client.py", line 1052, in _send_output
    self.send(msg)
  File "/usr/lib/python3.11/http/client.py", line 990, in send
    self.connect()
  File "/usr/lib/python3.11/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x745cb54d2fd0>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='mirror.sentries.org', port=80): Max retries exceeded with url: /sanesecurity/badmacro.ndb.sha256 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x745cb54d2fd0>: Failed to establish a new connection: [Errno -2] Name or service not known'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/fangfrisch/refresh.py", line 120, in refresh
    digest = get_digest(ci)
             ^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/fangfrisch/download.py", line 73, in get_digest
    download = _download(f'{ci.url}.{ci.check}', max_size, ci.connection_timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/fangfrisch/download.py", line 60, in _download
    response = _session.get(url, stream=True, timeout=timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='mirror.sentries.org', port=80): Max retries exceeded with url: /sanesecurity/badmacro.ndb.sha256 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection

[rseichter]

The errors are created inside the Requests library, a third party product. The message "Name or service not known" indicates a DNS resolver problem, not the inability to create a connection. If this happens to you often, maybe try using different resolvers? In case you do want to suppress those errors being logged, which I do not recommend, you can try setting log_level = FATAL in your config file.

[kitterma]

On April 10, 2024 9:39:50 PM UTC, Ralph Seichter @.***> wrote:

The errors are created inside the Requests library, a third party product. The message "Name or service not known" indicates a DNS resolver problem, not the inability to create a connection. If this happens to you often, maybe try using different resolvers? In case you do want to suppress those errors being logged, which I do not recommend, you can try setting log_level = FATAL in your config file.

It might be useful to catch the exception and make a clearer error message along these lines, so people know what to do.

Scott K

[rseichter]

@kitterma I am not a fan of catching and re-interpreting other software components' exceptions. Among other disadvantages, it makes it harder to look up the original error messages on the Internet. It doesn't help people if they have to go through me to find more information.

[kitterma]

On April 10, 2024 10:16:10 PM UTC, Ralph Seichter @.***> wrote:

@kitterma I am not a fan of catching and re-interpreting other software components' exceptions. Among other disadvantages, it makes it harder to look up the original error messages on the Internet. It doesn't help people if they have to go through me to find more information.

I understand the point, but particularly for applications (vice libraries), you can often add enough context for it to be worthwhile. Clearly, in this case, the native error was not enough.

[rseichter]

It was enough for silly old me. 😉 Also, if you have a look at the three exceptions raised in the traceback shown above, they are too generic to interpret programmatically. It takes a human to do that.