Enables multiple transports to bind to the same port

[mmasaki]

This change specifies UV_UDP_REUSEADDR and SO_REUSEPORT for the socket in Transport::UDP. In Linux, by combining the multicast address and SO_REUSEPORT, multiple transports across workers can bind to the same port. When distributing a stream to multiple workers, this eliminates the need to send the stream via unicast to each worker. This should be convenient for broadcasting.

Example:

const same_port = 1234;
const router1 = await worker1.createRouter(...);
const router2 = await worker2.createRouter(...);
transport1 = await router1.createPlainTransport({ listenIp: "224.0.0.1", comedia: true, port: same_port });
transport1 = await router2.createPlainTransport({ listenIp: "224.0.0.1", comedia: true, port: same_port }); // This does not cause an error.

[mmasaki]

@nazar-pc In Linux, multiple processes can receive the same packets simultaneously when binding UDP sockets to a multicast address with SO_REUSEPORT. This is a special behavior.

[nazar-pc]

I'm quite confident you're misunderstanding it. In case of both TCP and UDP it allows distribution of connections/datagrams across listeners, not broadcasting. Distribution means one datagram will be processed by one worker, another by another, all managed by the kernel.

A great example for UDP is DSN server. It would be odd to get multiple DNS responses to a single request, wouldn't it?

[mmasaki]

@nazar-pc The normal behavior is as you would expect. However, the behavior is special only when binding to a multicast address with SO_REUSEPORT. In this case, multiple processes can receive the same packets.

[nazar-pc]

Okay, I read about it more, and it only has the effect you expect on multicast addresses, not on any address (which this PR doesn't check), which makes this a very niche use case. And even then I'm not sure if some feedback will be necessary, which will not work properly in this case, but might be functional for plain transport specifically.

[ibc]

it only has the effect you expect on multicast addresses, not on any address (which this PR doesn't check), which makes this a very niche use case.

Exactly. This PR is unconditionally applying "reuse port" to the every UDP open port. This means that, if by accident the app binds into a non multicast address in same UDP port in 2 workers, the second transport won't fail but the behavior later will be completely erratic with UDP packets being randomly distributed to one worker or another. This is not an acceptable change since it prevents a legitimate user error from being noticed.

[ibc]

I don't know anything about multicast so I need to understand with an easy and detailed use case how this feature is supposed to behave. In which case we want to use a multicast address? What would happen if two clients send RTP packets to different transports in different workers in same multicast IP and port? Will each UDP packet reach both workers? If so each packet maybe ignored in one of those workers due to "unknown ssrc" or similar. And it would be worse in WebRtcTransports where clients will first send STUN and DTLS packets that require a response from server side.

[mmasaki]

I've updated the patch. It turns out that SO_REUSEPORT is not necessary for this purpose, so I've removed it. The patch now only adds UV_UDP_REUSEADDR in the case of a multicast address. With this change, multiple workers will be able to receive a stream on the same port. I believe this is useful for broadcast purposes and is not a niche use case.

[ibc]

@mmasaki I made some specific questions in my comment above: https://github.com/versatica/mediasoup/pull/1266#issuecomment-1857758624

I promise that we cannot merge something that we don't understand, so please answer those questions.

[mmasaki]

In which case we want to use a multicast address?

This is useful for distributing the same video/audio to multiple workers for live streaming purposes.

What would happen if two clients send RTP packets to different transports in different workers in same multicast IP and port?

This feature is intended to be used with comedia: true. If two clients send packets simultaneously, as before, the client whose packets arrive first will be choosed.

Will each UDP packet reach both workers?

Yes. That's why it is useful for broadcast purposes. This change will allow us to easily distribute video/audio to multiple workers using tools like ffmpeg or gstreamer.

Example:

same_port=1234
# we can distribute video/audio to workers with one simple command
ffmpeg \
    -re \
    -v info \
    -stream_loop -1 \
    -i ${MEDIA_FILE} \
    -map 0:a:0 \
    -acodec libopus -ab 128k -ac 2 -ar 48000 \
    -map 0:v:0 \
    -pix_fmt yuv420p -c:v libvpx -b:v 1000k -deadline realtime -cpu-used 4 \
    -f tee \
    "[select=a:f=rtp:ssrc=${AUDIO_SSRC}:payload_type=${AUDIO_PT}]rtp://${audioTransportIp}:${same_port}|[select=v:f=rtp:ssrc=${VIDEO_SSRC}:payload_type=${VIDEO_PT}]rtp://${videoTransportIp}:${same_port}"

If so each packet maybe ignored in one of those workers due to "unknown ssrc" or similar. And it would be worse in WebRtcTransports where clients will first send STUN and DTLS packets that require a response from server side.

This feature is intended to be used with plainTransport.

[ibc]

Problems of this PR are:

1. It's intended only for plain transport, but nothing in the code verifies it. A user may accidentally use same multicast IP for WebRTC transports and things would work erratically. So the way to do this should be by adding a specific new enableMulticast Boolean option in PlainTransportOptions in Node and Rust.
2. This PR is checking whether given address is multicast in PortManager.cpp. It should be a new Utils::IP::IsMulticast(addr) utility in Utils.hpp instead.
3. PortManager.bind() methods should have a new enableMulticast argument and it should only be set to true when called from PlainTransport.cpp constructor and only if PlainTransportOptions has enableMulticast, this is Linux and the given IP is verified to be multicast.

I'll leave this PR on hold and make those required changes when possible, unless you want to make them.

[nazar-pc]

I agree this is a potentially useful features, but it does need to be developed a bit further to be mergeable.

[mmasaki]

I have tried to resolve the issues. Could you please review it?

[mmasaki]

@ibc Thank you for reviewing. I have applied your suggestions.

[ibc]

This looks good. Just a bit of patience please since they are complex days and probably will merge this on next week or beginning of 2024.

[ibc]

@mmasaki could you please enable permissions for us to push to your branch?

[ibc]

Actually I'm reconsidering the way this PR is done. I don't think we need any special enableMulticast option just for PlainTransport. Instead we can add flags to TransportListenInfo struct and let the user pass socket flags. I'll write a PR.

[jmillan]

@mmasaki, how does this work for RTCP?. In multicast, will the RTCP generated by the worker properly reach the sender?

[ibc]

BTW from libuv docs https://docs.libuv.org/en/v1.x/udp.html#c.uv_udp_flags:

    /*
    * Indicates if SO_REUSEADDR will be set when binding the handle in
    * uv_udp_bind.
    * This sets the SO_REUSEPORT socket flag on the BSDs and OS X. On other
    * Unix platforms, it sets the SO_REUSEADDR flag. What that means is that
    * multiple threads or processes can bind to the same address without error
    * (provided they all set the flag) but only the last one to bind will receive
    * any traffic, in effect "stealing" the port from the previous listener.
    */
    UV_UDP_REUSEADDR = 4,

[ibc]

I'm working on this PR: https://github.com/versatica/mediasoup/pull/1291

[ibc]

I'm closing this PR in favour of already merged PR https://github.com/versatica/mediasoup/pull/1291. Thanks.