Instead of keeping running EC2 instances as buffer machine, we would only keep their respective EBS volumes up.
Needed changes
AMI:
current boot script automatically uses the largest disk it finds to mount the docker folder, needs to change to only target EBS
needs to set an EBS disk
currently large EC2s have a larger disk (up to 3.4TB which is free of charge and available to the users), need an equivalent EBS? when yes we need to parametrize this and define what sizes are needed
Autoscaling:
when an EC2 is started it received the "UserData" script that "pre-pulls" docker images such as s4l that is large,
instead of keeping X running buffer machines, it would stop them instead and terminate the ones that are above the buffer number,
it must wait until the pre-pulling took effect before stopping the machines and I think there is no way but SSH in order to know that and that is not very nice, investigate some other way (maybe start with a hard-coded delay)
it must handle the disks data (images will accumulate over time and fill the disk), how?
if we only stop instances, then we need to book keep the available stopped instances. what is the advantage over shutting them down?
we need to book keep the created volumes and ensure they do not accumulate
Concept
Instead of keeping running EC2 instances as buffer machine, we would only keep their respective EBS volumes up.
Needed changes
AMI:
Autoscaling: