search

elasticluster / elasticluster

Create clusters of VMs on the cloud and configure them with Ansible.
http://elasticluster.readthedocs.io/
GNU General Public License v3.0
336 stars 150 forks source link

On EC2, EBS volumes are not deleted on termination #353

Open nicoulaj opened 7 years ago

nicoulaj commented 7 years ago

When using EBS-backed images that do not have the "delete on termination" flag set, such as CentOS 6 official AMI (ami-edb9069e), the EBS instance root volume is not deleted on stop. Since elasticluster fully automates creation/destruction of the cluster, one would expect it overrides this, and guarantees stopping a node will release all associated resources.

Adding related Gitter discussion:

nicoulaj @nicoulaj 10:06 Is it normal that on EC2, elasticluster does not delete the instances EBS volumes on stop ? There is a "Delete on termination" flag when attaching an EBS volume to an instance, should elasticluster set it or is it due to the AMI I use ?

Riccardo Murri @riccardomurri 10:16 @nicoulaj I think ElastiCluster should set it, but maybe it doesn't

nicoulaj @nicoulaj 10:19 I see register_image has the flag, so it looks related to the image: http://boto.cloudhackers.com/en/latest/ref/ec2.html#boto.ec2.connection.EC2Connection.register_image

Riccardo Murri @riccardomurri 10:20 Hmmm... I would presume that sets a default, and then one can override it when creating a particular instance of an AMI?

nicoulaj @nicoulaj 10:21 I guess, but it looks a bit messy, see: tschellenbach/Snaptastic#2

Riccardo Murri @riccardomurri 10:23 It looks like one can specify that parameter at instance creation time using BlockDeviceMapping: http://boto.cloudhackers.com/en/latest/ref/ec2.html#module-boto.ec2.blockdevicemapping although it's not clear to me whether the devices in block_device_mapping=... are used in addition to the boot disk, or they override the params for the boot disk

nicoulaj @nicoulaj 10:34 I think it adds... This page explains it: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html On my Centos 6 AMI, I can see "Block Devices /dev/sda1=snap-b337027a:8:false:standard" So I guess if we wanted to handle it in elasticluster, it would have to query all devices after the instance is started, and change the flag...

Riccardo Murri @riccardomurri 10:39 looks like your guess is correct, given what theuser guide page says.

pescobar commented 7 years ago

I have also noticed the same issue during my tests in AWS

georg-rath commented 4 years ago

Mr @pescobar, long time no see! We seem to discover the same things, some time apart from each other :)

Did you find a solution for this? We are currently starting to use elasticluster to provision throwaway clusters for integrations tests and except this one thing, we are incredibly happy with it.

pescobar commented 4 years ago

hello Mr. @georg-rath !! We missed you during FOSDEM :)

Sorry but I have no solution for this issue. I only did some quick tests in AWS and I never used it in production so I I didn't invested time digging into the problem.