Server binding says "Address already in use"

[droberin]

cloned the code and tried server.py using default port and also port 2000 (as many others that are not in use).

Results:

$ python3 server.py --port 2000
INFO:pymidi.server:Data socket on 0.0.0.0:2001
INFO:pymidi.server:Control socket on 0.0.0.0:2000
INFO:pymidi.server:Data socket on :::2001
Traceback (most recent call last):
  File "server.py", line 157, in <module>
    server.serve_forever()
  File "server.py", line 108, in serve_forever
    self._init_protocols()
  File "server.py", line 100, in _init_protocols
    data_protocol = self._build_data_protocol(host, family)
  File "server.py", line 95, in _build_data_protocol
    data_socket.bind((host, self.port + 1))
OSError: [Errno 98] Address already in use

[mik3y]

Hmm, I wonder if this is a regression caused by #6 .. @garthwebb able to take look?

[garthwebb]

I just cloned a fresh version of the repo and set it up and I'm not able to repro this with the default port or 2000:

$ python pymidi/server.py
INFO:pymidi.server:Data socket on 0.0.0.0:5052
INFO:pymidi.server:Control socket on 0.0.0.0:5051
INFO:pymidi.server:Data socket on :::5052
INFO:pymidi.server:Control socket on :::5051

$ python pymidi/server.py --port 2000
INFO:pymidi.server:Data socket on 0.0.0.0:2001
INFO:pymidi.server:Control socket on 0.0.0.0:2000
INFO:pymidi.server:Data socket on :::2001
INFO:pymidi.server:Control socket on :::2000

Netstat doesn't show anything else? What OS is running?

[droberin]

I'm using Ubuntu 18.04 LTS, git clone (master branch) Using virtualenvs [python3 (3.6.6) / python2 (2.7.15rc1) ] Netstat doesn't show any of those ports listening (at least not before or after executing)

$ python2 pymidi/server.py
INFO:pymidi.server:Data socket on 0.0.0.0:5052
INFO:pymidi.server:Control socket on 0.0.0.0:5051
INFO:pymidi.server:Data socket on :::5052
Traceback (most recent call last):
  File "pymidi/server.py", line 157, in <module>
    server.serve_forever()
  File "pymidi/server.py", line 108, in serve_forever
    self._init_protocols()
  File "pymidi/server.py", line 100, in _init_protocols
    data_protocol = self._build_data_protocol(host, family)
  File "pymidi/server.py", line 95, in _build_data_protocol
    data_socket.bind((host, self.port + 1))
OSError: [Errno 98] Address already in use
$ netstat -anul | grep 5051
$ netstat -anul | grep 5052

No output from netstat

If I set a localhost IPv6 address to listen to it seems to work!! I just find out:

$ python pymidi/server.py -B ::1
INFO:pymidi.server:Data socket on 0.0.0.0:5052
INFO:pymidi.server:Control socket on 0.0.0.0:5051
INFO:pymidi.server:Data socket on ::1:5052
INFO:pymidi.server:Control socket on ::1:5051

Netstat

2>/dev/null netstat -antulp | grep python 
udp        0      0 0.0.0.0:5051            0.0.0.0:*                           10645/python        
udp        0      0 0.0.0.0:5052            0.0.0.0:*                           10645/python        
udp6       0      0 ::1:5051                :::*                                10645/python        
udp6       0      0 ::1:5052                :::*                                10645/python        

Doesn't seem to allow me to set it on :: or ::0

[droberin]

I think problem may be related to sysctl default configuration. As when setting an IPv6 address it tries to listen to IPv4 too, which is already in use.

I leave this github issue link from ejabberd https://github.com/processone/ejabberd/issues/984

[mik3y]

Ahh, that's a great clue! Cracks the mystery: Depending on the OS, and specifically its dualstack support, binding to an IPv6 address may automatically also bind to the equivalent IPv4 address; which we have just occupied..

I'll have to get back to this in a day or two, but I'm inclined to make the following changes to the Server class (and the demo server) to resolve this:

1. In Server, update to constructor to take an iterable of bind_addresses, each comprising a (ip, port) tuple.
   • When binding, Server must auto-detect protocol family, eg by testing socket.inet_pton(family, addr)
2. In the demo server, drop the flags --bind_host, --bind_port, and --bind_ipv6_host and replace with --bind_addr. Default this to 0.0.0.0:5051, and allow it to be specified multiple times.

This would make it possible to support concurrent IPv4 and IPv6 binding on both dualstack and non-dualstack (if those are the correct terms) OSes.

### Default (ipv4 only)
$ python pymidi/server.py --bind_addr=0.0.0.0:5051

### On a dualstack host (implicit ipv4 bind)
$ python pymidi/server.py --bind_addr=:::5051

### On a non-dualstack host (explicit ipv4 bind)
$ python pymidi/server.py --bind_addr=:::5051 --bind_addr=0.0.0.0:5051

Learn something new every day, I guess! @garthwebb you better be running that RPi on some really cool IPv6 network 😁

[mik3y]

Update: Went ahead and hacked out the changes above; see https://github.com/mik3y/pymidi/tree/mikey/py3 and specifically https://github.com/mik3y/pymidi/commit/41c0758bae373f61546ab1f42c9008739dc65504 .. Will PR tomorrow.

[garthwebb]

Great! I've been on a far flung Thanksgiving with little internet. I can give an extra pair of eyes to your PR but probably not until Wednesday.

On Sun, Nov 25, 2018 at 8:21 PM mike w notifications@github.com wrote:

Update: Went ahead and hacked out the changes above; see https://github.com/mik3y/pymidi/tree/mikey/py3 and specifically 41c0758 https://github.com/mik3y/pymidi/commit/41c0758bae373f61546ab1f42c9008739dc65504 .. Will PR tomorrow.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mik3y/pymidi/issues/8#issuecomment-441487486, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEbfu3g10s37fIswF7SrZ2KgorZfZWMks5uyzPxgaJpZM4Yr_p0 .

[droberin]

I initially though that IPv6 was optional, so tried to set it to "None", "null", "no", etc... So, if a tiny device has an IPv4 interface only it would work.

Thanks for your work, mates.