search

membraneframework-labs / membrane_videoroom

Apache License 2.0
42 stars 8 forks source link

High CPU usage #190

Closed daviaws closed 7 months ago

daviaws commented 1 year ago

Using a Droplet with 4 Basic-CPU from Digital Ocean. I opened a room with 7 peers. And it's giving me a High CPU usage profile. I do not know much about webrtc. But I know there is a peer to peer connection. So, if the stream is p2p. Why am I having so high usage on server side? There is any way to avoid this?

Plus 1: For 1 Core Droplet I can handle 3 connections with almost 100% CPU usage. Plus 2: For 4 Core Droplet I can handle 7 connections with almost 80% CPU usage. The growth isn't linear... Plus 3: Even with 12 UDP ports opened, a range from 50000-50012. I cannot open an 8º connection... it crashes server side. I was facing issues with 6 ports opened which I cannot exceed 3 connections.

image

image

mickel8 commented 1 year ago

Hi @daviaws, I don't think there is much we can do. On the server side it's (n-1)^2 connections so when you have 7 people we send 36 streams inside as every person is sent to 6 other people. Regarding 12 UDP ports, seems like a bug :thinking: Do you have any stacktrace?

daviaws commented 1 year ago

I see. It's ok to accept this complexity to forwarding communication. Although I find a very high consumption profile. Aka 4 vcpus optimized for CPUs and 10 peers in 1 room, the application consumed 50% of the CPU. image Sometimes trying to allocate more peers, some of the peers froze and stop to send or receive stream... reload caused some of other peers to froze. I found this strange as I still had hardware to consume.

I do not have a stack trace right now for the udp ports. But I'm always in need to allocate more ports than peers.

If you have some interesting logs you want me to send, please, tell me which logs and I'll provide.

But I'm curious about CPU usage and the system architecture. Can I have some clusterized approach so I can allocate more and more hardware with a K8S architecture and Elixir cluster in an easy way with current membrane code? The advantage of autoscaling hardware in cost and efficiency would be great. This video shows OTP and K8S working together with a set of tools as libcluster, horde and CRDT: https://www.youtube.com/watch?v=nLApFANtkHs

mickel8 commented 1 year ago

@daviaws Regarding frozen peers etc. Do you have any server or client logs? Are there any errors in web console? I wouldn't expect you need more ports than you have peers but if you allocate more ports, does everything work correctly?

When it comes to the K8S and libcluster. We already merged libcluster-based load balancing to the Jellyfish but we haven't released it yet. It's a simple load balancing mechanism where if you have a cluster of Jellyfishes and you want to create a new room, it will be created on the Jellyfish with the lowest load.

We don't support splitting one room across multiple nodes right now :slightly_frowning_face: