Agressive Auto Scaling in AWS - 1 -

mekya commented 1 year ago

It's known that auto scaling does not respond fast when there is a too much demand. Here is a typical basic scenario:

Steps

Have a single server 2 cores in an auto scaling ORIGIN group
Have a single server 2 cores in an auto scaling EDGE group
Publish a live stream
Play the stream with 2K viewers

For sure, 2 cores cpu cannot handle 2K viewers. Expectation is to have a solution that to scale the instances within 3 minutes to support 2K viewers and there is no bad user experience. Here are more details:

Confirm that the servers cannot receive more demands that they can serve.
Users always have good audio/video quality - No robotic audio/no video freeze due to load performance problems
Server may send 'highResourceUsage' but handle that on the viewer side gracefully
Users can wait 3 minutes in total to start the playing after live stream has started
It will be better to have solutions with and with and without warm pools but we can start with warm pools

Thank you so much pushing us for this issue @Ashraf

muratugureminoglu commented 1 year ago

Hi,

Since the AutoScale structure is triggered based on CloudWatch metrics, it will take some time for a 2 Core instance (we can say about 110-120 viewers, which means 17+1 instances) to be triggered by AutoScale for an aggressive 2K viewers. But I tried to do some tuning below, hope it helps.

1. If you are using simple monitoring, Cloudwatch metrics are created every 5 minutes on the EC2 instance. This will extend the triggering time of a new instance in AutoScale accordingly. If you use Detailed monitoring, you can decrease this time by up to 1 minute[1]. You can find if it's active or not in EC2 > Launch Configuration > Your Launch Configuration > Advance Configuration.

2. You can create a CloudWatch rule and then add it to AutoScaling as a Policy (The old policy should be deleted). In this CloudWatch alarm, if you set the datapoint as 1 and keep the Period time for 1 minute and reduce the Warm Up time from 300 seconds to 10 seconds, the new instance opening time will be around 2 minutes.