zeromq / libzmq

ZeroMQ core engine in C++, implements ZMTP/3.1
https://www.zeromq.org
Mozilla Public License 2.0
9.74k stars 2.36k forks source link

ZMQ_LAST_ENDPOINT doesn't contain port for udp transport #4661

Open WaelKarkoub opened 2 years ago

WaelKarkoub commented 2 years ago

I'm new to pyzmq but I believe I found a bug with bind_to_random_port. I ran this script to install zmq and pyzmq

#!/usr/bin/env bash

# example script for installing libzmq and pyzmq with draft support

# 1. install libzmq with draft enabled
export ZMQ_VERSION=4.3.4
export PREFIX=${PREFIX:-/usr/local}
export PYZMQ=${PYZMQ:-pyzmq}

set -ex
echo "installing libzmq to $PREFIX"
wget https://github.com/zeromq/libzmq/releases/download/v${ZMQ_VERSION}/zeromq-${ZMQ_VERSION}.tar.gz -O libzmq.tar.gz
tar -xzf libzmq.tar.gz
cd zeromq-${ZMQ_VERSION}
./configure --prefix=${PREFIX} --enable-drafts
make -j && make install

# install pyzmq with drafts enabled
# --install-option disables installing pyzmq from wheels,
# which do not have draft support

echo "installing ${PYZMQ}"
export ZMQ_PREFIX=${PREFIX}
export ZMQ_DRAFT_API=1

pip install -v --no-binary pyzmq --pre ${PYZMQ}

cat << END | python3
import sys
import zmq
print('python: %s' % sys.executable)
print(sys.version)
print('pyzmq-%s' % zmq.__version__)
print('libzmq-%s' % zmq.zmq_version())
print('Draft API available: %s' % zmq.DRAFT_API)
END

then trying to run some_socket.bind_to_random_port("udp://*") I get the following error

File "/usr/local/lib/python3.8/dist-packages/zmq/sugar/socket.py", line 413, in bind_to_random_port
    return int(port_s)
ValueError: invalid literal for int() with base 10: '*'

line 411 in sugar/socket.py, url = cast(bytes, self.last_endpoint).decode('ascii', 'replace'), i'm getting url = *:*.

Let me know if there's a way I can contribute to this bug.

minrk commented 2 years ago

I think this is a bug in libzmq, unless I'm mistaken. When you bind to a random port, it's really doing:

s.bind('tcp://127.0.0.1:*')
url = s.last_endpoint
# tcp://127.0.0.1:12345

and parsing the port out of the LAST_ENDPOINT property. However, when you access LAST_ENDPOINT after a udp bind, it doesn't return the resolved port:

In [23]: s.bind('tcp://127.0.0.1:*');

In [24]: s.last_endpoint
Out[24]: b'tcp://127.0.0.1:55021'

In [25]: s.bind('udp://127.0.0.1:*');

In [26]: s.last_endpoint
Out[26]: b'127.0.0.1:*'

I don't know if this is on purpose, but it means bind_to_random_port's default behavior can't work until libzmq fixes that.

If you specify an explicit port range, pyzmq has to use its own logic to pick a random port (which can require retries, as it picks a port before trying to bind), in which case it works:

In [27]: s.bind_to_random_port('udp://*', min_port=10101, max_port=12345)
Out[27]: 10611