Scale-in capability

[ababaian]

Let me know what you think of this set up.

• Workers now will auto-shutdown after 10 consecutive scheduler wait/retry events to prevent idle. That is ~100 seconds.
• Shutdown will adjust the ASG DesiredCapacity -1 immediately prior to shutdown. There is a global 300s cooldown on ASG changes being enacted.
• Workers now have EC2 describe and Autoscaling adjustment permissions
• If a worker has a running process, it will enact "scale-in protection" on itself. When all workers are done on an instance, this is removed.

---

This still needs testing to confirm these functions work.

[ababaian]

This was probably the most frustrating bit of bash debugging I've ever run into. We squash and never speak of it again.