iann0036 / former2

Generate CloudFormation / Terraform / Troposphere templates from your existing AWS resources.
https://former2.com
MIT License
2.24k stars 269 forks source link

Why did Former2 try to CreateServiceLinkedRole #117

Open payne911 opened 4 years ago

payne911 commented 4 years ago

After pressing Scan, two API calls appeared in CloudTrail:

1) SearchProvisionedProducts, which seems legit to me. 2) The one below. One second after the first one. Not sure why it happened on us-east-1, too, since all I have is on ca-central-1.

image

Why exactly did it try to do that?

Also, this was ultimately generated after selecting Include all Resources (see the typo with ${IAMGroup}istrative):

  IAMServiceLinkedRole2:
    Type: "AWS::IAM::ServiceLinkedRole"
    Properties:
      AWSServiceName: "support.amazonaws.com"
      Description: !Sub "Enables resource access for ${EC2PrefixList.OwnerId} to provide billing, ${IAMGroup}istrative and support services"

Additionally, since I scanned (three times I think?), I have a forecasted monthly cost of 0.02$.

I've also no idea why it apparently discovered LoadBalancers, and some EC2PrefixList which point to S3 and DynamoDB considering I never spinned up any of those things. Same for ElastiCacheParameterGroup, and OpsWorksUserProfile, and ElasticBeanstalkApplication, and ApiGatewayAccount, and AthenaWorkGroup, and GlueDataCatalogEncryptionSettings...


Otherwise, thank you for Former2 ! It's an awesome service. Saved me days of work by generating a pretty nice >600 LOC template.

iann0036 commented 4 years ago

Hey @payne911,

Thanks for reaching out!

The CreateServiceLinkedRole is unusual, so I'll investigate why that occurred, however it's likely a side-effect of calling a separate Get/List API. As roles relate to IAM, and IAM is a global service, it will show up as us-east-1 in call logs. Same thing for Route 53, WAF, CloudFront etc.

The ${IAMGroup}istrative is essentially the template logic trying to be too smart. It will attempt some substitution where it believes it is appropriate. As it doesn't have English language context, it will occasionally do this in error.

There may be some costs to resource retrieval which is unavoidable - hopefully it provides more value than it costs.

I'm unsure of the Load Balancer and Prefix List results, however the majority of the other resources such as ApiGatewayAccount or GlueDataCatalogEncryptionSettings are account-wide settings which exist the moment you create an AWS account. Similarly, an OpsWorksUserProfile is created at the same time you create an IAM user. I currently don't have logic to differentiate these between their default states.

Happy to take a look at your template to see if there are some resources that are included that shouldn't be there.

payne911 commented 4 years ago

Thanks for your insightful answer!

Since my IAMGroup is called Admin, I can understand why it substituted it in ${IAMGroup}istrative. Kind of funny, even!

There may be some costs to resource retrieval which is unavoidable - hopefully it provides more value than it costs.

This should definitely be part of your documentation (or maybe I just missed it). What exactly could end up costing money from making a bunch of List and Get API calls ?

Happy to take a look at your template to see if there are some resources that are included that shouldn't be there.

Don't feel obligated, but that would be very kind of you! Here are the initially generated template, and my current refactoring of the template.

The refactoring is still a pretty big WIP, and you'll notice a few # todo comments a bit all over. Overview of my todos:

That Volume Snapshot is the biggest mystery: I don't recall ever asking for snapshots because those are usually paid-for (due to required storage). This is apparently something that's supposed to help me figure out where it comes from: image

I do notice how DestinationAmi is the one my EC2 is based on, but cannot find anything about that SourceAmi. And in any case, I've no idea why there's a snapshot associated with all of this.

Ultimately, for reference, this is more or less the stack I want to achieve.

Unfortunately for me, I'm a bit scared of running the template to test it out because I do not want to be charged just for testing the launch of the stack (which is supposed to be a free stack). And I'm also afraid about how well CloudFormation will be able to destroy the whole stack properly without leaving anything behind afterward. (This is my first time messing with IaC.)

iann0036 commented 4 years ago

Hey @payne911,

There is a bunch of services which charge for Get/List calls (example), so I'll make an update to the README to reflect this.

I notice your sample templates have been pulled from Pastebin before I got to them, so feel free to shoot me an email with the details.

As for CloudFormation itself, it is generally very good at cleaning up its own resources when you delete the stack, however consequential resources like CloudWatch logs would remain so make sure to consider this. A billing alarm will catch overages, however you should plan to be billed a dollar or two for small things like this.

payne911 commented 4 years ago

There is a bunch of services which charge for Get/List calls (example), so I'll make an update to the README to reflect this.

Maybe you should provide an option to exclude such services' requests?

I notice your sample templates have been pulled from Pastebin before I got to them, so feel free to shoot me an email with the details.

Sent you an email. Very kind of you, thank you!

iann0036 commented 4 years ago

Hey @payne911,

Got your email, thanks for that!

The majority of the erroneous resources come from default resources - I'll make some effort to try to reduce these, however some are impossible to differentiate from the real thing (hence the "Experimental" tag on the feature).

Maybe you should provide an option to exclude such services' requests?

It would be very difficult to confidently remove billable actions from the tool as pricing does change often, will consider this though.

On your original issue regarding the CreateServiceLinkedRole call for support, I can't actually replicate this as this service-linked role is non-deletable and is created by default for both new standalone and Organizational AWS accounts. In any case, you may want to create the SLR yourself as it is likely recommended.

Is this a somewhat old account?

payne911 commented 4 years ago

Okay, I see.

Is this a somewhat old account?

Almost a year old, with nothing but this one stack that has been created a few months ago.

iann0036 commented 4 years ago

Cheers for that @payne911,

I've made adjustments to make everything considered a default resource (that is, something that is present immediately after account creation) not be returned.

I've also flat out ignored the Support SLR. It has weird behaviours :)

I'm also investigating why the Global Accelerator SLR seems to be created as part of list calls.

Thanks again for all of your info.

payne911 commented 4 years ago

Very responsive maintainer and cool project: thanks to you!

iann0036 commented 4 years ago

Hey @payne911,

Just returning back to the original question you had, I've discussed this with the AWS service teams and it does seem that calling globalaccelerator.ListAccelerators will trigger the internal call to iam.CreateServiceLinkedRole to make the Global Accelerator SLR.

It seems this is an uncontrollable side-effect of making that call, which is required to list the accelerators within the account.