SaxyPandaBear
commented
4 years ago Issues regarding scaling out are mostly going to depend on the websocket connection management. This is already enumerated in #32. With Lambda and DynamoDB being serverless, we don't have to worry so much about those. The real issue is going to be managing all of the websocket connections. Closing this out since that is the last item to deal with on this front.
Scaling out is a potential growing pain. The way the POC is going so far, it is one WebSocket connection for one channel's event topic. For a million connections (unlikely, but a possibility), this becomes a major problem.
Scaling up is another problem. We have to make sure that we handle requests as they come, so as not to block things. We can't allow our app to be the bottleneck for processing, or Twitch will cut off the connection to the websocket if we are too slow. If we can't queue up songs quickly, then we are sunk.
I'll try to keep this as a living doc whenever I get a new idea, but will close out once I decide on a solution and run with it.
Brain Dump 1 - Scale Out:
I envision a solution where we would essentially put a load balancer on top of our connections. The problem will be that a user's stream will not be live 24/7. They will not constantly be receiving requests, so it is possible that we will have many idle connections. What I want to do is have some sort of GUI (how else will we direct users to authorize our usage of their Twitch/Spotify assets?) and in their browser, we will send the request to a remote server:
When I think about it, I don't think we need the webapp to really persist. It is just going to be firing a request to a remote orchestrator. The problem will be the Spotify connection. This is a complete unknown right now, and this needs to be ironed out. If we can get access to queue songs in their own Spotify app via web APIs, then we are solid. If, however, we are required to connect directly to a web player, then this complicates things as it creates the need for a persistent connection through their browser.
Brain Dump 2 - Scale Up:
This is actually a pretty dumb approach, but hinges entirely on how we implement the Spotify integration. That aside, the actual work of processing a Twitch event and queueing a Spotify song should be pretty straightforward. What isn't straightforward is when there is latency in the Spotify API, which under stress could cause a backlog of queue requests. It could even rate limit us. If we use an internal buffer/queue, then we can mitigate this. We could just throw things into the queue, and have a polling mechanism to read from the queue on a frequency, like every minute, and do a batch request (hopefully) to enqueue a bunch of songs at once. However, if we can't do that, then we'll have to up the frequency and only request one at a time.