unexpected EPROTOTYPE returned by sendto on MAC OSX

77941739-77db-4f23-a630-1e1eaf0f132a commented 6 years ago

BPO	33450
Nosy	@ronaldoussoren, @ned-deily, @asvetlov, @1st1, @racitup, @erlend-aasland

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/ronaldoussoren' closed_at = None created_at = labels = ['OS-mac', 'type-bug', 'expert-IO', 'expert-asyncio'] title = 'unexpected EPROTOTYPE returned by sendto on MAC OSX' updated_at = user = 'https://github.com/racitup' ``` bugs.python.org fields: ```python activity = actor = 'erlendaasland' assignee = 'ronaldoussoren' closed = False closed_date = None closer = None components = ['macOS', 'IO', 'asyncio'] creation = creator = 'racitup' dependencies = [] files = [] hgrepos = [] issue_num = 33450 keywords = [] message_count = 4.0 messages = ['316328', '316353', '327724', '396486'] nosy_count = 8.0 nosy_names = ['ronaldoussoren', 'ned.deily', 'asvetlov', 'yselivanov', 'racitup', 'Eamonn Nugent', 'erlendaasland', 'Daniel King'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue33450' versions = ['Python 3.5', 'Python 3.6'] ```

77941739-77db-4f23-a630-1e1eaf0f132a commented 6 years ago

The following exception is raised unexpectedly on macOS versions 10.13, 10.12 & 10.11 at least. It appears to be macOS specific (works okay on Linux).

Further information can be found at the following links: https://github.com/benoitc/gunicorn/issues/1487 http://erickt.github.io/blog/2014/11/19/adventures-in-debugging-a-potential-osx-kernel-bug/

[2017-03-20 00:46:39 +0100] [79068] [ERROR] Socket error processing request.
Traceback (most recent call last):
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 66, in handle
    six.reraise(*sys.exc_info())
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/six.py", line 625, in reraise
    raise value
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 56, in handle
    self.handle_request(listener_name, req, client, addr)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/ggevent.py", line 152, in handle_request
    super(GeventWorker, self).handle_request(*args)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 129, in handle_request
    six.reraise(*sys.exc_info())
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/six.py", line 625, in reraise
    raise value
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/workers/async.py", line 115, in handle_request
    resp.write(item)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/http/wsgi.py", line 362, in write
    util.write(self.sock, arg, self.chunked)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gunicorn-19.7.0-py3.6.egg/gunicorn/util.py", line 321, in write
    sock.sendall(data)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gevent-1.2.1-py3.6-macosx-10.12-x86_64.egg/gevent/_socket3.py", line 418, in sendall
    data_sent += self.send(data_memory[data_sent:], flags)
  File "/Users/ahmad/Projects/Side-Gigs/sa7beh-app/venv/lib/python3.6/site-packages/gevent-1.2.1-py3.6-macosx-10.12-x86_64.egg/gevent/_socket3.py", line 391, in send
    return _socket.socket.send(self._sock, data, flags)
OSError: [Errno 41] Protocol wrong type for socket

ronaldoussoren commented 6 years ago

The second link contains an explanation of what's going on, and that this is unexpected behaviour of the macOS kernel.

I'm not sure what we could do about this, the blog post explains that this error can happen when send(2) is called while the socket is teared down. A possible workaround (based on reading the blog post and without fully analysing the side effects) is to treat EPROTOTYPE the same as EGAIN in the python wrappers for send/write/sendto.

That's easier said than done though, socketmodule.c indirectly calls socket functions through a helper function that does some bookkeeping (including handling EGAIN) and is used for more than just the send calls.

af3f2516-8e0c-4dcd-9303-62a889e653ba commented 6 years ago

Fwiw, this also happens in asyncio. Theoretically, I *think* you could wrap a try-except as a monkey patch.

My stack trace is:

Fatal write error on socket transport
protocol: <RequestHandler connected>
transport: <_SelectorSocketTransport fd=163 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 761, in write
    n = self._sock.send(data)
OSError: [Errno 41] Protocol wrong type for socket

I can try to PR the fix mentioned by @ronaldoussoren, though it might take me a bit. I'm happy to do whatever to get this fixed, though, since it's flooding my terminal with several hundred errors every time it happens...

erlend-aasland commented 3 years ago

python / cpython

unexpected EPROTOTYPE returned by sendto on MAC OSX #77631