WordPress / openverse

Openverse is a search engine for openly-licensed media. This monorepo includes all application code.
https://openverse.org
MIT License
244 stars 195 forks source link

Thumbnail request raises "OSError: [Errno 107] Transport endpoint is not connected" #655

Closed AetherUnbound closed 10 months ago

AetherUnbound commented 1 year ago

Sentry link

https://openverse.sentry.io/share/issue/1764849effe1498c85c4138a3bb105e6/

Description

This seems to be a sparse issue, we've only received 5 events over the last few days. The stack trace looks like this:

OSError: [Errno 107] Transport endpoint is not connected
  File "urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "urllib3/connection.py", line 456, in connect
    cert = self.sock.getpeercert()
  File "gevent/_ssl3.py", line 440, in getpeercert
    self._check_connected()
  File "gevent/_ssl3.py", line 365, in _check_connected
    self.getpeername()
  File "gevent/_socketcommon.py", line 560, in getpeername
    return self._sock.getpeername()

ProtocolError: ('Connection aborted.', OSError(107, 'Transport endpoint is not connected'))
  File "requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "urllib3/connection.py", line 456, in connect
    cert = self.sock.getpeercert()
  File "gevent/_ssl3.py", line 440, in getpeercert
    self._check_connected()
  File "gevent/_ssl3.py", line 365, in _check_connected
    self.getpeername()
  File "gevent/_socketcommon.py", line 560, in getpeername
    return self._sock.getpeername()

ConnectionError: ('Connection aborted.', OSError(107, 'Transport endpoint is not connected'))
  File "catalog/api/utils/photon.py", line 72, in get
    upstream_response = requests.get(
  File "requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "requests/adapters.py", line 547, in send
    raise ConnectionError(err, request=request)

For all I can tell, this is an issue deep in the bowels of the request process. I've looked for "Transport endpoint is not connected" in both urllib3 and gevents and neither gave me any results. All of the StackOverflow results for this seem to point to socket usage, which is not something we're dictating by hand when we issue this request.

Reproduction

The URLs for each event don't reproduce the issue for me.

sarayourfriend commented 10 months ago

Closing as stale. The sentry issue doesn't exist any more and we haven't seen it at all recently as far as I can tell. If it comes up again we can open a new issue and prioritise it.