Support for multicast and IPv6

mdavidsaver commented 2 years ago

Added support for multicast search and beacons over IPv4 and IPV6.

User configuration

EPICS_PVA_ADDR_LIST and friends accept expanded entries of the form <IP>[:port][,TTL#][@iface]. Where IP may be IPv4 dotted quad or an IPv6 address enclosed in square brackets. Some examples:

192.168.1.1 as before is a unicast address
224.0.2.3,255@192.168.1.1 specifies a multicast group 224.0.2.3 reached via the interface with address 192.168.1.1 where UDP packets sent will have TTL of 255.
ff02::42:1,1@br0 an IPv6 link scope multicast group ff02::42:1 reached via the interface br0, using a TTL of 1. (this being a rough functional equivalent to an IPv4 broadcast address)

Note that the @ syntax is overlapping with the IPV6 specific # syntax for specifying interfaces. Currently this is always resolved in favor of the @ syntax.

Sockets

By default a TCP socket bound to 0.0.0.0 will be "upgraded" to an IPv6 socket bound to :: allowing incoming connections over either IP version.
Sockets bound to a specific IPv4 or IPv6 address will of course be limited to that IP version.
UDP sockets are limited to either IPv4 or IPv6 as it is not possible for an AF_INET6 socket to interact with IPv4 multicast groups through 4to6 address mapping.

Current State

At present IPv6 UDP handling must be explicitly enabled.

mdavidsaver commented 2 years ago

Some example configurations I'm testing.

All IPv4 multicast

PVXS_LOG=*=DEBUG \
EPICS_PVA_AUTO_ADDR_LIST=NO \
EPICS_PVA_ADDR_LIST=224.0.1.1,1@127.0.0.1 \
EPICS_PVAS_INTF_ADDR_LIST='0.0.0.0 224.0.1.1,1@127.0.0.1' \
./example/O.linux-x86_64-debug/mailbox box

PVXS_LOG=*=DEBUG \
EPICS_PVA_AUTO_ADDR_LIST=NO \
EPICS_PVA_ADDR_LIST=224.0.1.1,1@127.0.0.1 \
./bin/linux-x86_64-debug/pvxget box

All IPv6 multicast

PVXS_LOG=*=DEBUG \
EPICS_PVA_AUTO_ADDR_LIST=NO \
EPICS_PVA_ADDR_LIST=ff02::42:1,1@br0 \
EPICS_PVAS_INTF_ADDR_LIST='0.0.0.0 ff02::42:1,1@br0' \
./example/O.linux-x86_64-debug/mailbox box

PVXS_LOG=*=DEBUG \
EPICS_PVA_AUTO_ADDR_LIST=NO \
EPICS_PVA_ADDR_LIST=ff02::42:1,1@br0 \
./bin/linux-x86_64-debug/pvxget box

AppVeyorBot commented 2 years ago

:white_check_mark: Build pvxs 1.0.701 completed (commit https://github.com/mdavidsaver/pvxs/commit/3fcaa4afef by @mdavidsaver)

kasemir commented 2 years ago

I've tested this against the softIocPVA and pvget/put/monitor from base-7.0.6 and the ongoing PVA_IPv6 branch of the Java server/client. All combinations of the following to "work", except for the impossible case of pure IPv6 clients (address list only has IPv6 entries) talking to IPv4 server.

Server: IPv4 softIocPVA, IPv4 'mailbox', IPv6 'mailbox', IPv4 Java server, IPv6 Java server Client: IPv4 pvget/put/monitor, IPV4 pvxget/put/monitor, IPv6 pvxget/put/monitor, IPv4 Java client, IPv6 Java client

There is a problem, however, at least on Mac OS, with both the PVXS and Java server for IPv6 when another softIocPVA for IPv4 is already running. When starting a softIocPVA instance, netstat -an | fgrep 5075 shows one tcp4 socket in LISTEN state:

tcp4       0      0  *.5075                 *.*                    LISTEN

When now starting either a PVXS and Java server for IPv6 , netstat shows this:

tcp46      0      0  *.5075                 *.*                    LISTEN     
tcp4       0      0  *.5075                 *.*                    LISTEN

Both the PVXS and Java server create a 'universal' TCP socket that can handle both IPv4 and IPv6. They succeed to bind that socket to port 5075, in spite of softIocPVA already having bound that port to an IPv4 socket. When a client now searches for a channel on the PVXS or Java server, they will reply to the search with "talk to me on TCP 5075", whereupon the client ends up with a connection to the softIocPVA server, which then replies "not found" to the create-channel request.

When starting another PVXS or Java server, those will find that TCP 5075 is already bound, but the first PVXS or Java server after softIocPVA fails to detect that TCP 5075 is already in use for tcp4.

kasemir commented 2 years ago

For the Java implementation, turns out that TCP server sockets default to the 'tcp46' type. From JDK15 on there is API to create a 'tcp4'-only server socket, which could be used to verify binding to the port, and it does detect if the port is already in use. But that would require using JDK15 or higher, plus the 'tcp6' socket that we need will again turn into a 'tcp46' type.

My current workaround is to simply try connecting to 127.0.0.1 port 5075 (respectively the configured PVA server port). If it succeeds, we know the port is already in use by a tcp4 server that's either bound to *:5075 or specifically to 127.0.0.1:5075. We could still miss an existing tcp4 server on port 5075 because our connection simply takes too long and times out (1 second?), or it's bound to some specific local address instead of wildcard or localhost.

AppVeyorBot commented 2 years ago

:white_check_mark: Build pvxs 1.0.710 completed (commit https://github.com/mdavidsaver/pvxs/commit/7bbb973d27 by @mdavidsaver)

mdavidsaver commented 2 years ago

There is a problem, however, at least on Mac OS ...

Ah Kay, you've lifted the scab on another significant OS dependent behavior... I'm working on some test coverage to probe the behavior of bind()ing sockets to both 0.0.0.0 and :: with the same port. What I see so far is:

Linux doesn't allow this for TCP (w/o IPV6_V6ONLY), or UDP (w/o REUSEADDR/PORT).
Windows and RTEMS5+libbsd allow both to be bound in either order.
OSX allows binding 0.0.0.0 and then ::, but errors on the second if the order is reversed.

I'm also surprised to see OSX and RTEMS/libbsd behaving differently. I'm under the impression that both are ~recent forks of a *BSD network stack.

So it looks like avoiding conflicts w/ IPv4 implementations is going to be different for each target.

I'm going to try the following as a starting point for TCP listeners:

Linux. Need to magically "promote" 0.0.0.0 to :: since there can be only one.
Windows/RTEMS/OSX. Need to bind both, starting with 0.0.0.0 to detect a conflict.

I'm also going to have to add a test case which tests binding to the wildcard address(es). Currently all high level testing uses a configuration w/ 127.0.0.1.

(fun fact, Docker doesn't configure IPv6. So container builds can't run tests of IPv6)

AppVeyorBot commented 2 years ago

:white_check_mark: Build pvxs 1.0.732 completed (commit https://github.com/mdavidsaver/pvxs/commit/2f34f6af9a by @mdavidsaver)

mdavidsaver commented 2 years ago

@kasemir I think I've addressed the bind() inter-op issue w/ OSX. Can you re-test?

AppVeyorBot commented 2 years ago

:white_check_mark: Build pvxs 1.0.735 completed (commit https://github.com/mdavidsaver/pvxs/commit/ba13db9258 by @mdavidsaver)

kasemir commented 2 years ago

Yes, now good.

To test, I'm starting a softIocPVA which binds to tcp4 *.5075, then starting the pvxs mailbox example.

Before, mailbox would bind to tcp46 *.5075, not noticing that port is already bound as tcp4. Now, starting the mailbox example detects "Server unable to bind port 5075, falling back to 0.0.0.0:52648".

kasemir commented 2 years ago

Another improvement: The build tended to fail in one of the later test code stages. Now all builds without errors on Mac OS X.

AppVeyorBot commented 2 years ago

:x: Build pvxs 1.0.748 failed (commit https://github.com/mdavidsaver/pvxs/commit/30a971ae1e by @mdavidsaver)

AppVeyorBot commented 2 years ago

:white_check_mark: Build pvxs 1.0.762 completed (commit https://github.com/mdavidsaver/pvxs/commit/5a5cbefee9 by @mdavidsaver)

mdavidsaver commented 2 years ago

Updated to fix an issue with automatic configuration which was preventing beacons from being sent by default on Linux.

mdavidsaver / pvxs