Running on kubernetes without hostnetwork=true theoretically possible ?

[St4NNi]

Hi,

I´m planning to run the containerized version of BBB in docker and want to give kubernetes a try. Sadly I am severely limitted in my environment regarding publicly available open ports and I don´t want to make direct use of the host network.

In my understanding the requirement for the large range of open udp ports and the use of the host network could be circumvented by running the whole BBB application stack including the TURN server in a single shared docker network and exposing only the necessary ports like 80/443 and some UDP-Ports etc. via proxy. Is this assumption correct, or am I missing something ?

I know that running BBB like that is not recommended and I´m fine with performance issues. I just wanted to ask for feedback if this approach is theoretically possible or if it is a complete waste of time (and why). If there is a chance to make it happen i will give it a try and am happy to share my experiences with the project.

Thanks in advance

[GyR4uk]

Hello. I'm interested in your idea. Regarding ports: they must be open: 443/tcp 16384-32768/upd, but this range can be reduced, but it is not safe

[St4NNi]

Thanks for your reply. This is the official requirement and I have read about it. But I thought the whole point about the TURN server is the relaying part where a whole range of UDP ports can be funneled through a NAT via a limited number of accessible ports. Usually this is an external separate server that does this, but in my idea I would use it to funnel the whole udp traffic to my container internal network. This way I should not need the whole 16384-32768/upd range but only a few for the TURN server, like one or two.

[GyR4uk]

I think you will not be able to let all traffic through the turn server.

[St4NNi]

I really would appreciate some technical insights on why you think this is not possible, maybe someone else has some input on this too ? Nonetheless I am now experimenting by myself and will post some personal experiences here soon.

[ThomasAppDev]

Did you get it working without turn-Server? Or are already actively working on it?

[St4NNi]

I made some progress, but first needed to convert the docker-compose files and images to proper kubernetes deployment yamls including the needed shared volumes, configmaps and publicly available images. Unfortunately this takes a lot of time.

I think it is not possible without a TURN server. Ingress and services in Kubernetes are not really build for larger port-ranges exceeding ~50-100 ports, so either you limit your ports severely and are restricted to 5-10 users or you have to use the hostnetwork.

[alangecker]

I feel a little bit overwhelmed by the discussion and miss knowledge about kubernetes, so I think I can't contribute a lot, but would be happy if someone makes is working! :)

Maybe an potential issue I encountered while initially while trying out port forwarding: I think IPv6 packets were forwarded by docker via IPv4, leading to freeswitch not working because it requires that packets get delivered to the corresponding listeners. Thats also why there is a conditional forwarding in nginx: https://github.com/bigbluebutton/docker/blob/develop/mod/https/site.conf#L5

So simply don't forget to try out both, IPv4 and IPv6 :)

[GyR4uk]

Let's solve the problem sequentially, you are required to indicate what problems you are facing. And if it's not difficult, describe what you have already achieved.

I can share my experience in specific problems.

If you deploy software by application, now I think that the bottleneck will be freeswitch (load is not balanced normally).

If you deploy a lot of BBBs that do not depend on each other, the question arises in balancing and autoscaling.

As for proxying all traffic through the TURN server, this is not the best solution since the channels of the operators on which the server is running are limited.