search

ketzacoatl / compute-demo

HA compute / data processing cluster on AWS, built with Terraform, Packer, Nomad, Consul, and SaltStack.
3 stars 0 forks source link

dynamic auto-scaling with AWS #25

Open ketzacoatl opened 6 years ago

ketzacoatl commented 6 years ago

Use AWS instead of Ladder. It's now even easier, see - https://aws.amazon.com/blogs/aws/aws-auto-scaling-unified-scaling-for-your-cloud-applications/.

The task here: follow the notes in the blog post above to setup the new auto-scaling capabilities with our demo cluster, then run a bunch of instances of the sysbench task to get the cluster hot.. AWS should scale to the details in our policy.

Note that, the new auto-scaling capabilities/features are not available in GovCloud, so we would need to a) continue with this issue for the immediate moment, and b) create a new ticket that provides dynamic auto-scaling with AWS using cloud watch/etc (the previous method, and what's available in GovCloud).

kerscher commented 6 years ago

Should this ticket be changed to reflect “b) create a new ticket that provides dynamic auto-scaling with AWS using cloud watch/etc (the previous method, and what's available in GovCloud).” instead?

As for which metrics to use, I am going with this strategy:

  1. Create an unified scaling with the new method for “availability” option
  2. Analyze the generated CloudWatch Alarms
  3. Codify CloudWatch Alarms into a module that GovCloud can use, emulating the predefined policy