BUG: (?) zmq.SERVER socket creation failing with zmq.asyncio, in zmq.backend.cython.checkrc._check_rc and AttributeError: FD attribute is write-only

[spchamp]

This is a pyzmq bug

• [?] This is a pyzmq-specific bug, not an issue of zmq socket behavior. Don't worry if you're not sure! We'll figure it out together.

What pyzmq version?

25.1.2

What libzmq version?

4.3.4 (bundled)

Python version (and how it was installed)

Python 3.11.2, via openSUSE devel:languages:python factory

OS

openSUSE Leap 15.5

What happened?

When testing an application with ZMQ client and server sockets using zmq.asyncio, I'm seeing the message AttributeError: FD attribute is write-only during server socket initialization.

After subsequent testing, this error occurs similarly with the draft/client-server.py example ported for zmq.asyncio.

After building pyzmq from source, I'm able to view a traceback presenting an earlier error, in a call within the cffi sections of pyzmq. The traceback is included below.

Code to reproduce bug

# https://github.com/zeromq/pyzmq/blob/main/examples/draft/client-server.py

import asyncio as aio
import time
import zmq
import zmq.asyncio as zasync

async def run_test():
    ctx = zasync.Context.instance()

    url = 'tcp://127.0.0.1:5555'
    server = ctx.socket(zmq.SERVER)
    server.bind(url)

    for i in range(10):
        client = ctx.socket(zmq.CLIENT)
        client.connect(url)
        await client.send(b'request %i' % i)
        msg = await server.recv(copy=False)
        print(f'server recvd {msg.bytes!r} from {msg.routing_id!r}')
        server.send_string('reply %i' % i, routing_id=msg.routing_id)
        reply = await client.recv_string()
        print('client recvd %r' % reply)
        await aio.sleep(0.1)
        client.close()

    server.close()
    ctx.term()

if __name__ == "__main__":
    aio.run(run_test())

Traceback, if applicable

Traceback (most recent call last):
  File "/project/examples/env/lib64/python3.11/site-packages/zmq/sugar/attrsettr.py", line 55, in __getattr__
    return self._get_attr_opt(upper_key, opt)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/examples/env/lib64/python3.11/site-packages/zmq/sugar/attrsettr.py", line 67, in _get_attr_opt
    return self.get(opt)
           ^^^^^^^^^^^^^
  File "zmq/backend/cython/socket.pyx", line 503, in zmq.backend.cython.socket.Socket.get
  File "zmq/backend/cython/socket.pyx", line 270, in zmq.backend.cython.socket._getsockopt
  File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Invalid argument

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/examples/examples/zmq_cs_async.py", line 32, in <module>
    aio.run(run_test())
  File "/usr/lib64/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/project/examples/examples/zmq_cs_async.py", line 13, in run_test
    server = ctx.socket(zmq.SERVER)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/project/examples/env/lib64/python3.11/site-packages/zmq/sugar/context.py", line 362, in socket
    s: ST = socket_class(  # set PYTHONTRACEMALLOC=2 to get the calling frame
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/examples/env/lib64/python3.11/site-packages/zmq/_future.py", line 233, in __init__
    self._fd = self._shadow_sock.FD
               ^^^^^^^^^^^^^^^^^^^^
  File "/project/examples/env/lib64/python3.11/site-packages/zmq/sugar/attrsettr.py", line 61, in __getattr__
    raise AttributeError(f"{key} attribute is write-only")
AttributeError: FD attribute is write-only

More info

Using Python installed from OpenSUSE devel:languages:python repository, openSUSE Build System: https://build.opensuse.org/project/show/devel:languages:python

In this newer installation for pyzmq, the installation uses the bundled libzmq from pyzmq, built with GCC at installation time, presumably using the same compiler settings as the openSUSE python build.

I'm not certain which libzmq was used in the earlier installation. Both were installed from PyPI for this same pyzmq version. The second installation was created with pip install -vv --force --no-binary=pyzmq pyzmq

[spchamp]

This might be similar to the following https://github.com/zeromq/pyzmq/issues/1139

[spchamp]

If client, server sockets may not available with zmq.asyncio, will look at other approaches. There's always pub/sub lol

[spchamp]

Reading up more about this in the links from nr. 1139, if there may be socket types in zmq that don't use file descriptors in so much (??) and the application is itself using asyncio for coroutine dispatch, it might be just as well to use the synchronous API within these coroutines?

Do I understand that there might not be any blocking on FD I/O in the process or the thread, with these sockets specifically? client/server, radio/dish, and with inproc transports even?

The application is already a mashup of synchronous and async approaches LoL and it has something of a threading model, not worried about style I guess

[minrk]

Correct, the "threadsafe" group of socket types do not expose zmq.FD, so are therefore incompatible with event-loop integration via the zmq.asyncio subclass. You can still use any sync sockets and avoid blocking the event-loop by always using DONTWAIT in your send/recv and handle the waiting yourself with e.g. asyncio.sleep:

import asyncio
import zmq

async def recv_multipart(sock, flags=0, copy=True, poll_interval=0.1):
    while True:
        try:
            return sock.recv_multipart(flags | zmq.DONTWAIT, copy=copy)
        except zmq.Again:
            # can't wait natively, try again in a bit
            await asyncio.sleep(poll_interval)

That's a coroutine that will block until recv returns, but won't block the event loop because send/recv with DONTWAIT always return immediately. You could also do a smarter wait with exponential backoff, jitter, timeout, etc.

You can also put blocking socket calls in a background thread via ThreadPoolExecutor, which is usually not safe, but it is specifically these "threadsafe" sockets that have this problem, so might be fine. It would be safe as long as all socket methods occur in the same thread.

Closing here as this is the same issue as #1139, but feel free to continue there.