search

hubs-community / mozilla-hubs-ce-chart

MIT License
14 stars 9 forks source link

Details about port range used for coturn and dialog services #7

Open vincentfretin opened 5 months ago

vincentfretin commented 5 months ago

I couldn't easy find reading the README and the chart the port range used for coturn and dialog services.

FYI in previous aws CloudFormation template, we had those settings and I'm using those settings in my own helm chart for a project using aframe and previous janus sfu.

coturn turnserver.conf min-port=49152 max-port=51609

and janus rtp_port_range = "51610-65535"

now hubs is using dialog instead of janus.

In https://github.com/hubs-community/mozilla-hubs-ce-chart it says 35000 - 60000 to be open.

For dialog I think this is the configuration https://github.com/mozilla/dialog/blob/257315f8a418c756fc466c26a8bcc43d6c4d568e/config.js#L62 rtcMinPort : process.env.MEDIASOUP_MIN_PORT || 40000, rtcMaxPort : process.env.MEDIASOUP_MAX_PORT || 49999 I don't see any MEDIASOUP_MIN_PORT MEDIASOUP_MAX_PORT variables in the helm chart so I think it's using 40000-49999 range.

For coturn, I see it's using https://hub.docker.com/r/doginal/mozilla-hubs-coturn with no github repo I could find. Looking directly inside the image:

docker run -it doginal/mozilla-hubs-coturn bash
cat entrypoint.sh
min-port=49152
max-port=51609

so using 49152-51609 range

Isn't it an issue to have overlapping port range here for the two services if the two pods ends up on the same instance?

Doginal commented 5 months ago

Hey, With hostNetwork set to true only a single instance/pod can run of that type because only a single port exists. The LB is not a part of this as it's using the host's network (public ip) to do the peering.

I had issues with the preview version of Hubs ce which is why I built the image. I may swap it back to Mozilla's version but it still wasn't working the last time I tried it.

vincentfretin commented 5 months ago

With hostNetwork set to true only a single instance/pod can run of that type because only a single port exists. The LB is not a part of this as it's using the host's network (public ip) to do the peering.

Yes I know that. But two pods using hostNetwork: true (here one pod dialog and one pod coturn) can be on the same instance/vm/worker of the pool, using the worker public ip. I'm not sure what the behavior of coturn allocating a port in the configured port range and mediasoup already allocated the same port, or vice versa. It's a hard to see issue unless you are under heavy loads and exhausted most of the ports in the range.

So it would probably best to set MEDIASOUP_MAX_PORT=49151 instead of the default 49999 to be sure this is not an issue.

Doginal commented 5 months ago

It's not an issue. They use different ports. (I have them running on the same node no issues) Dialog uses 4443. CoTurn uses port 5349 TCP and UDP 35000-60000.

vincentfretin commented 5 months ago

Dialog is a sfu, 4443 is only used for the signaling part, and a udp port is allocated for each webrtc connection, in the range 40000-49999 from the config I see in the dialog repo. For users that don't need a TURN server, no udp port is used for coturn.

vincentfretin commented 5 months ago

For the coturn image, for maintenance and trust, it would be probably best if we can use the official coturn image that is built automatically on new release and when alpine version change https://github.com/coturn/coturn/releases

Doginal commented 5 months ago

@vincentfretin I believe the base Moz image was based on the official coturn image.

I have no issues with swapping it to a default. The hubs one is creating the config at build time and i would like it as a configmap.