Suggestion: A/V conference beacon mode, for more scalability

[drequivalent]

I stole half of this idea from Jitsi Videobridge. How do we do an audio/video converencing? Well, one way is send streams from everyone to everyone. See this picture (sorry for poor drawing, didn't have time for something more pleasant than LO Draw): https://files.equivalent.me/public.php?service=files&t=c0b4892548ad6d2c5502bcf57800121d This mostly works fine, but this will require more trafic as more people join the conference. Both upstream and downstream. Then, as we hit certain quantity, we hit a bottleneck: all clients suddenly run out of bandwidth. Mostly, it's upstream bandwidth that is problematic, as the internet servce providers provide downstream more readily than upstream. The other way, is to centralize the conference. As in, all the clients send their streams to only one node, and then the said node sends all the streams it has to everyone else. This will let us save on vauable upstream bandwidth. But then again, if the cantral node fails, it will ruin everything for all the clients in a conference. So, what should we do? I think, we should implement the mechanism of selecting such central node dynamically. The way I see it is the following: Everyone in a conference pings for other clients accessibility, and polls for upstream bandwidth. And, broadcasts these values to everyone else in a conference. The node who is equally accessible to everyone else and has the most bandwidth, becomes a Beacon - it says the others that it is ready to handle all their upstream traffic, so there's no need to send to anyone else. Schematically, it looks like this: https://files.equivalent.me/public.php?service=files&t=d040be5e5380adf426cd3c08e23d1467 So, all the clients still have fat downstream. But that's much less of an issue, than fat upstream. For stability reasons, it also makes sense to choose a fallback beacon node, that will be ready to serve when the main one goes offline, so everybody quickly switches to it. And also may be beacon re-election every now and then. In other words, there's a lot to discuss, but I think saving upstream bandwidth will allow Tox conferences to be more scalable.

[srkunze]

I like this idea.

[cebe]

:+1: would you mind adding the pictures directly to the github issue? you can drag them into the comment field or click the link below the text field. Your links run into timeout for me...

[czarkoff]

I don't think central node is necessary. Clients may broadcast requests to all nodes in the group call, asking to limit outgoing stream to certain amount. Recievers then pick the strictest limit they've seen and decide on their way to enfoce such limit (eg. apply more aggressive compression or resize the video).

Pros:

• no single point of failure,
• works when all members of the call have similar bandwidth issues,
• avoids degrading video quality (central node needs to reencode lossy compressed video),
• avoids performance bottleneck (central node would need a lot of CPU performance to decode and reencode everything on fly) and
• allows more flexibility for clients.

Cons:

• clients will still need to decode many incoming streams, while central node could combine them into one single stream to decode.

[srkunze]

@czarkoff Why is re-encoding necessary? I thought simply re-broadcasting to the group members would suffice.

[czarkoff]

@srkunze Re-broadcasting would not reduce bandwidth usage for anyone - only increase it for central node. Without re-encoding the whole mechanism is useless.

Wait, is A/V data supposed to be delevered in unicast to every node?

[weedy]

You're missing the point of this discussion.

You have a group chat going with say 6 people. Two or three of those have like 10/1 DSL, another two have 15/7, and the last one or two people have 25/15. Currently the 10/1 dudes would have to drop down to like 240p to fit 5 streams into there upload limit.

The option we are discussing here is once a client (probably the 10/1 dudes) passes some arbitrary threshold of I dunno, latency plus quality downgrades. Them that client can ask the other clients to relay his stream for him. Maybe now he can fit 720p into his upstream and one of the 25/15 dudes takes over sending his stream to the remaining people.

Of course being a relay should have lower priority then sending your own stream. And we need to be able to hand off relating top other clients if we leave. Etc etc many things to work out.

This might also open up the ability for "hosting" a group chat somewhere. Say the 10/1 knows his connection is shit. So he gets a 100/100 vps and installs some tox bootstrap/proxy node something. Now has can just bounce all his streams off that, and maybe it gets advertised in his group chats as a relay and his friends also use it saving every one of his friends in the chat upstream bandwidth.

None of this has any need to reencode anything. it's strictly leveraging unused bandwidth at other endpoints.

[czarkoff]

I was under impression that group chat should work as multicast, meaning that each of participants sends data only once, in one stream.

[srkunze]

@czarkoff So, the Internet service providers perform the necessary stream distribution?

[weedy]

Multicast packets are not routed on the open internet.

[GrayHatter]

@irungentoo close as stale and replaced by new groupchats

[ProMcTagonist]

New groupchats have gossip and automagic scaling for AV?

[GrayHatter]

it'll suggest clients scale, but doesn't enforce it