ekristen / aws-nuke

Remove all the resources from an AWS account
https://ekristen.github.io/aws-nuke/
MIT License
39 stars 7 forks source link

Filter for all resource types #76

Closed YuriGal closed 6 months ago

YuriGal commented 6 months ago

We're tagging all of our resources we want to keep from nuking with a specific tag.

Is it possible to do something like this in the config:

presets:
  keep-tag:
    filters:
      *:
        - property: 'tag:nuke.keep'
          value: 'true'
ekristen commented 6 months ago

Thanks for the issue! Welcome!

I'm actually in the process of writing support for a global filter. https://github.com/ekristen/libnuke/pull/37

It will only work if the resource has the tag property defined, but as soon as I have it completed and tested I'll be doing another beta release and will reference this issue so you can test it out.

ekristen commented 6 months ago

I've merged it into libnuke, but I need to update this tool and write some documentation, I'll try and get to that as soon as I can, but it won't be today.

ekristen commented 6 months ago

This is now on the beta.17 release -- https://github.com/ekristen/aws-nuke/pull/77

All feedback and testing would be appreciated. While I've taken great pains to not cause bugs or break things. Take care in testing.

YuriGal commented 6 months ago

Will give it a spin, thanks! This should probably reduce size of our config 10 times if not more.

ekristen commented 6 months ago

I'm also looking into YAML anchors as anothe way to reduce config sizes and a subcommand to help test configs. Please remember this is beta and feedback is most welcome.

ekristen commented 6 months ago

@YuriGal just following up on this. any chance to test?

YuriGal commented 6 months ago

Sorry, was diverted to other stuff, will test today!

ekristen commented 6 months ago

No worries!

YuriGal commented 6 months ago

Hey Erik, unfortunately it looks like nuke in its current state for won't work for us, due to some changes from the original nuke. Some changes we can live with (e.g. everywhere in our CI we have option -c for config, but we can switch that to --config), but also option to assume role in an another account --assume-role-arn seems to be missing to. In our use-case nuke runs in a codebuild in a service account targeting client accounts, so we need this.

ekristen commented 6 months ago

I can definitely add -c back that was an oversight. The assume role should definitely be there. Are you getting an error? If you can let me know your exact usage with things sanitized I'll test and get back to you.

YuriGal commented 6 months ago

Basically this is our command

aws-nuke -c nuke-config.yml --quiet --force --force-sleep 3 --assume-role-arn arn:aws:iam::\<account-id>:role/<role-name> $FLAGS

$FLAGS is either --no-dry-run or unset. This works in the original nuke, in this one it throws

FATA[0000] flag provided but not defined: -assume-role-arn

for some reason with one dash.

I just realized - I probably need to use run command? Original nuke just accepts parameters.

YuriGal commented 6 months ago

Ok, I got past CLI huddles, the command is definitely works. But now I am getting

"failed get caller identity: InvalidParameter: 1 validation error(s) found.\n- minimum field size of 2, AssumeRoleInput.ExternalId.\n"
YuriGal commented 6 months ago

Ok, apparently external ID is needed now, I've added it. Assume role work, but I don't think filter did, unless I am doing this wrong. I have filter set as

    filters:
      __global__:
      - property: tag:Baseline"
        value: "true"

but in logs I see

global - IAMPolicy - arn:aws:iam::xxxxxxxxxxxxxxx:policy/DetectiveInvitationsPolicy - [ARN: "arn:aws:iam::xxxxxxxxxxxxxxxx:policy/XXXXXXXXXXXXXXXXXX", Name: "XXXXXXXXXXXXXXXXX", Path: "/", PolicyID: "XXXXXXXXXXXXXXXXXXX", tag:Baseline: "true"] - would remove
ekristen commented 6 months ago

@YuriGal thanks for this, I'll test and get back to you ASAP. It should be optional but sounds like there's a logic issue. I have plans to overhaul the AWS in the future, I'm opening issues up and referencing them now.

ekristen commented 6 months ago

@YuriGal it looks like your __global__ has a extra quote " on the end of the property based on what you pasted in.

YuriGal commented 6 months ago

Damn. Looks like you're right, sorry. I got too reliant on VS Code YAML syntax highlighter, usually it warns about these mismatches, but now it didn't. Will give this another go.

YuriGal commented 6 months ago

It works as designed! I will continue to play with this version, but it looks like it's working.

ekristen commented 6 months ago

@YuriGal great!

FYI - hyphen options for long form aren't supported with the CLI library I use, -- are for long form. - are for short form like -c that's why the -assume-role-arn triggered an error. I've got https://github.com/ekristen/aws-nuke/pull/80 opened and will be merged soon to address the issues you brought up here.

YuriGal commented 6 months ago

Awesome thanks!

I think I might've discovered another potential problem, this one is about actual run, but I will open a separate issue.

ekristen commented 2 months ago

:tada: This issue has been resolved in version 3.0.0 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket:

YuriGal commented 2 months ago

Woohoo! Thanks!