CommandRunner not terminating ec2 instances on CloudFormation deployment

[james-gibbons-lark]

Sometimes the ec2 instances will delete sometimes they won't after a deployment. Has anyone else seen this? We have hundreds of old running ec2 instances I've just noticed

[shantgup]

Sorry for the delay in getting back to you.

So CommandRunner will always make the TerminateInstances API call, however, many users have SCP policies or tagging-requirements that will deny the API call and the EC2 instance will remain in the account until deleted manually.

This is the reason why CommandRunner also checks to see if it has permissions for the TerminateInstances API call before it even creates the EC2 instance.

Were you using the new DisableTerminateInstancesCheck parameter?

I also recommend that you check CloudTrail's Event History and filter by EventName: TerminateInstances and look for the failed API calls made by CommandRunner. This will help you troubleshoot further and figure out why the API calls failed.

Thank you and let me know if you have any questions.

[james-gibbons-lark]

@shantgup We are still having issues with ec2 instances not deleting. I found some logs in the log group awsutility-cloudformation-commandrunner-logs2 that appear to be related to the management of the instances. There is an error on the [DELETE] invoking handler... Failed to execute remote function: {null (Service: AWSSimpleSystemsManagement; Status Code: 400; Error Code: ParameterNotFound; Request ID: njico2n4-4kkz13w-d6b4-470f-b92a; Proxy: null)}

Also, not sure if this might cause issues, but we are running multiple command runner commands in a single cloud formation template. So each CF deployment is spinning up multiple ec2 instances