search

gruntwork-io / cloud-nuke

A tool for cleaning up your cloud accounts by nuking (deleting) all resources within it
https://gruntwork.io/
MIT License
2.76k stars 354 forks source link

Bug - Not waiting long enough for Transit Gateway resources to nuke #624

Closed zachreborn closed 7 months ago

zachreborn commented 9 months ago

Description

When Transit Gateway resources exist, cloud nuke is not waiting long enough nor retrying on the nuking of these resources. The result of which is that depdendent resources are also unable to be nuked.

Example:

  WARNING  This program sends telemetry to Gruntwork. To disable, set DISABLE_TELEMETRY=true as an environment variable

# AWS Resource Query Parameters
┌────────────────────────────────────────────────────────────────────┐
| Query Parameter         | Value                                    |
| ------------------------------------------------------------------ |
| Target Regions          | 17 regions (too many to list all)        |
| Target Resource Types   | 58 resource types (too many to list all) |
| List Unaliased KMS Keys | false                                    |
└────────────────────────────────────────────────────────────────────┘

 INFO  Found 1 cloudwatch-loggroup resources in us-east-2
 INFO  Found 1 vpc resources in us-east-2
 INFO  Found 1 ec2_dhcp_option resources in us-east-2
 INFO  Found 3 eip resources in us-east-2
 INFO  Found 1 kmscustomerkeys resources in us-east-2
 INFO  Found 3 nat-gateway resources in us-east-2
 INFO  Found 1 transit-gateway-attachment resources in us-east-2
 INFO  Found 1 transit-gateway resources in us-east-2
 INFO  Done searching for resources

# Found AWS Resources
┌───────────────────────────────────────────────────────────────────────────────┐
| Resource Type              | Region    | Identifier                           |
| ----------------------------------------------------------------------------- |
| cloudwatch-loggroup        | us-east-2 | transit_vpc-vpc-flow-logs            |
| vpc                        | us-east-2 | vpc-0cbea1a06894aa854                |
| ec2_dhcp_option            | us-east-2 | dopt-0e79c92f7f9509195               |
| eip                        | us-east-2 | eipalloc-02585f8cdc9762df4           |
| eip                        | us-east-2 | eipalloc-014a61fe67a87e9d9           |
| eip                        | us-east-2 | eipalloc-0dbac1573f001a031           |
| kmscustomerkeys            | us-east-2 | 80f9954f-a3c0-4f8d-8edc-dc0d1f36fd0d |
| transit-gateway-attachment | us-east-2 | tgw-attach-075bef627a1dc65ab         |
| transit-gateway            | us-east-2 | tgw-0a76506bcdae2b6bb                |
└───────────────────────────────────────────────────────────────────────────────┘

 WARNING  THE NEXT STEPS ARE DESTRUCTIVE AND COMPLETELY IRREVERSIBLE, PROCEED WITH CAUTION!!!

Are you sure you want to nuke all listed resources? Enter 'nuke' to confirm (or exit with ^C) : nuke
  ERROR   Failed to nuke vpc with err: DependencyViolation: The subnet 'subnet-0cc2eddfc8edb8b2e' has dependencies and cannot be deleted.
            status code: 400, request id: b9a44cff-61ea-4c09-8597-7e7cb1c0dbb8
 INFO  Successfully deleted DHCP option dopt-0e79c92f7f9509195.

┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
| Identifier                           | Resource Type                    | Deleted Successfully                        |
| vpc-0cbea1a06894aa854                | VPC                              | ❌ DependencyViolation: The subnet 'subnet- |
| --------------------------------------------------------------------------------------------------------------------- |
| dopt-0e79c92f7f9509195               | ec2_dhcp_option                  | ✅                                          |
| --------------------------------------------------------------------------------------------------------------------- |
| tgw-attach-075bef627a1dc65ab         | Transit Gateway                  | ✅                                          |
| --------------------------------------------------------------------------------------------------------------------- |
| tgw-0a76506bcdae2b6bb                | Transit Gateway                  | ❌ IncorrectState: tgw-0a76506bcdae2b6bb ha |
| --------------------------------------------------------------------------------------------------------------------- |
| transit_vpc-vpc-flow-logs            | CloudWatch Log Group             | ✅                                          |
| --------------------------------------------------------------------------------------------------------------------- |
| eipalloc-014a61fe67a87e9d9           | Elastic IP Address (EIP)         | ✅                                          |
| --------------------------------------------------------------------------------------------------------------------- |
| eipalloc-0dbac1573f001a031           | Elastic IP Address (EIP)         | ✅                                          |
| --------------------------------------------------------------------------------------------------------------------- |
| 80f9954f-a3c0-4f8d-8edc-dc0d1f36fd0d | Key Management Service (KMS) Key | ✅                                          |
| --------------------------------------------------------------------------------------------------------------------- |
| eipalloc-02585f8cdc9762df4           | Elastic IP Address (EIP)         | ✅                                          |
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Second run:

 WARNING  This program sends telemetry to Gruntwork. To disable, set DISABLE_TELEMETRY=true as an environment variable

# AWS Resource Query Parameters
┌────────────────────────────────────────────────────────────────────┐
| Query Parameter         | Value                                    |
| ------------------------------------------------------------------ |
| Target Regions          | 17 regions (too many to list all)        |
| Target Resource Types   | 58 resource types (too many to list all) |
| List Unaliased KMS Keys | false                                    |
└────────────────────────────────────────────────────────────────────┘

 INFO  Found 1 vpc resources in us-east-2
 INFO  Found 1 transit-gateway resources in us-east-2
 INFO  Done searching for resources

# Found AWS Resources
┌─────────────────────────────────────────────────────┐
| Resource Type   | Region    | Identifier            |
| --------------------------------------------------- |
| vpc             | us-east-2 | vpc-0cbea1a06894aa854 |
| transit-gateway | us-east-2 | tgw-0a76506bcdae2b6bb |
└─────────────────────────────────────────────────────┘

 WARNING  THE NEXT STEPS ARE DESTRUCTIVE AND COMPLETELY IRREVERSIBLE, PROCEED WITH CAUTION!!!

Are you sure you want to nuke all listed resources? Enter 'nuke' to confirm (or exit with ^C) : nuke

┌────────────────────────────────────────────────────────────────┐
| Identifier            | Resource Type   | Deleted Successfully |
| tgw-0a76506bcdae2b6bb | Transit Gateway | ✅                   |
| -------------------------------------------------------------- |
| vpc-0cbea1a06894aa854 | VPC             | ✅                   |
└────────────────────────────────────────────────────────────────┘

Commands Run

cloud-nuke aws --exclude-resource-type iam-role --exclude-resource-type iam-service-linked-role --exclude-resource-type iam-group --exclude-resource-type cloudtrail --exclude-resource-type guardduty --exclude-resource-type security-hub --exclude-region global

Work Around

Re-running cloud-nuke will subsequently nuke out the TGW and VPC subnets.

james03160927 commented 8 months ago

Will look into this @zachreborn .