Distributed Streaming System

[Lakshya2610]

144 #142 #141 #140

Refer to the issues above to get an idea of how the system works. Main Features,

• Every client is a node in the tree with root as the host.

• When a new client connects, backend provides client with list of possible hosts in order of closest to the root. Client picks a host and connects and is added to the tree at the right position.

• When a non-root node disconnects, its child nodes reconnect to different nodes in the tree and bring all their child nodes with them.

• Whenever a node leaves, it reports to the backend of disconnect, and the tree is updated appropriately. If due to some reason, the client fails to report that its leaving, its parent node also reports the disconnection on RTC connection failure to make sure the Tree is always in the right state.

Fixes #71

I've added a checkbox on the extension to toggle this system for a room and have marked it "Experimental". This system needs a lot of testing to understand the different kind of scenarios that could occur. It works as of now with the limited testing I was able to do. But since the goal of this system is to scale, we need to test this with high no. of clients to understand pros and cons of this system

Next Steps for this:

• Fix Open issues related to this

• Implement a queuing system to make sure multiple clients don't connect at the same time as this can cause overloading of nodes beyond their max capacity

[Lakshya2610]

Forgot to mention, Since there are a lot of changes on every end of the app, I've made sure that all 3 parts of the app work just fine with older version of of one another as long as you don't use the new features (Trying to create a room with distributed streaming with older backend code would cause the backend to crash. But this isn't a big deal because the new version of backend works with older version of extension/client, so we can update the backend right away)

[Lakshya2610]

If you want to test the disconnect functionality, the easiest way would be to change the max client per node size to 1 on the backend and connect a few devices/tabs. This would essentially create a linked list. Then try removing a node in between to test how the system handles a node leaving. Just a heads-up, when a node leaves, all of its children nodes (and their children nodes) experience a tiny stutter in playback due to the gap in re-connection process. It's pretty minimal most of the time but it's there.

[Lakshya2610]

The reconnection process for a failure hasn't been tested yet. I just implemented it. I'm not sure how to simulate WebRTC connection failures

[Lakshya2610]

Codacy has raised many issues related to generic object injection. It's blocking the PR merge at the moment. We can think of a way to fix it or wait until we move our code over to Typescript, which would take care of it automatically.

https://web.archive.org/web/20150430062816/https://blog.liftsecurity.io/2015/01/15/the-dangers-of-square-bracket-notation

One way we could do it in standard JavaScript is to create a class to hold the list of rooms instead of using an Object.

[ArsalaBangash]

Codacy has raised many issues related to generic object injection. It's blocking the PR merge at the moment. We can think of a way to fix it or wait until we move our code over to Typescript, which would take care of it automatically. https://web.archive.org/web/20150430062816/https://blog.liftsecurity.io/2015/01/15/the-dangers-of-square-bracket-notation

One way we could do it in standard JavaScript is to create a class to hold the list of rooms instead of using an Object.

I think that would be best then. It's a slight pain but it'd greatly improve our code quality score and make the transition to Typescript more graceful.

[ArsalaBangash]

I think we've done a good job addressing a bunch of code quality issues. There's still quite a bit of generic object injection we have to address, so I've opened up a separate issue for it. For now, I'll merge this PR with admin privelages.