aws-cloudformation / cloudformation-coverage-roadmap

The AWS CloudFormation Public Coverage Roadmap
https://aws.amazon.com/cloudformation/
Creative Commons Attribution Share Alike 4.0 International
1.11k stars 56 forks source link

Validate AWS::CUR::ReportDefinition in template in all regions #1825

Open iainelder opened 1 year ago

iainelder commented 1 year ago

Name of the resource

Other

Resource name

AWS::CUR::ReportDefinition

Description

1565 explains that AWS::CUR::ReportDefinition is available only in us-east-1.

I accept that, but I think that you could improve the validation behavior for the resource in other regions.

The CUDOS deployment guide provides a template that conditionally creates the resource. If the stack is in us-east-1 the template creates the resource directly. Otherwise the template creates a Lambda that makes a cross-region request to create the CUR.

In practice the template works outside us-east-1. I have created a stack from the template in eu-central-1.

When I call ValidateTemplate on the same file, I get an error.

An error occurred (ValidationError) when calling the ValidateTemplate operation: Template format error: Unrecognized resource types: [AWS::CUR::ReportDefinition]

Instead I expect the template to pass validation.

Other Details

Extract from CloudFormation template:

####
# Local CUR 
####

  ## Deploy CUR nativly via CFN resource if we are in us-east-1
  LocalCurInSource:
    Type: AWS::CUR::ReportDefinition
    Condition: DeployCURViaCFNInSource
    DependsOn:
      - SourceS3BucketPolicy
    Properties:
      AdditionalArtifacts:
        - ATHENA
      AdditionalSchemaElements:
        - RESOURCES
      Compression: Parquet
      Format: Parquet
      RefreshClosedReports: True
      ReportName: !Ref ResourcePrefix
      ReportVersioning: OVERWRITE_REPORT
      S3Bucket: !If [ IsDestinationAccount, !Ref DestinationS3, !Ref SourceS3 ]
      S3Prefix: !Sub "cur/${AWS::AccountId}"
      S3Region: !Ref AWS::Region
      TimeUnit: HOURLY

  LocalCurInDestination:
    Type: AWS::CUR::ReportDefinition
    Condition: DeployCURViaCFNInDestination
    DependsOn:
      - DestinationS3BucketPolicy # Conditional DependsOn is not supported, so we need 2 resources
    Properties:
      AdditionalArtifacts:
        - ATHENA
      AdditionalSchemaElements:
        - RESOURCES
      Compression: Parquet
      Format: Parquet
      RefreshClosedReports: True
      ReportName: !Ref ResourcePrefix
      ReportVersioning: OVERWRITE_REPORT
      S3Bucket: !If [ IsDestinationAccount, !Ref DestinationS3, !Ref SourceS3 ]
      S3Prefix: !Sub "cur/${AWS::AccountId}"
      S3Region: !Ref AWS::Region
      TimeUnit: HOURLY

  # Deploy CUR via lambda due to missing cfn resource definition
  # AWS::CUR::ReportDefinition outside us-east-1
  CURinUSEAST1:
    Type: Custom::CURCreator
    Condition: DeployCURViaLambda
    Properties:
      ServiceToken: !GetAtt CIDLambdaCURCreator.Arn
      BucketPolicyWait: !If [ IsDestinationAccount, !Ref DestinationS3BucketPolicy, !Ref SourceS3BucketPolicy ]
      ReportDefinition:
        AdditionalArtifacts:
          - ATHENA
        AdditionalSchemaElements:
          - RESOURCES
        Compression: Parquet
        Format: Parquet
        RefreshClosedReports: True
        ReportName: !Ref ResourcePrefix
        ReportVersioning: OVERWRITE_REPORT
        S3Bucket: !If [ IsDestinationAccount, !Ref DestinationS3, !Ref SourceS3 ]
        S3Prefix: !Sub "cur/${AWS::AccountId}"
        S3Region: !Ref AWS::Region
        TimeUnit: HOURLY

###########################################################################
# Lambda CUR Creator: used to create cur from outside us-east-1
###########################################################################

  CIDLambdaCURCreatorRole: #Execution role for the custom resource CIDLambdaAnalyticsExecutor
    Type: AWS::IAM::Role
    Condition: DeployCURViaLambda
    Properties:
      Path:
        Fn::Sub: /${ResourcePrefix}/
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Policies:
        - PolicyName: "ExecutionDefault"
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
            - Effect: Allow
              Action:
              - logs:CreateLogStream
              - logs:PutLogEvents
              - logs:CreateLogGroup
              Resource:
              - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${ResourcePrefix}-CID-CURCreator"
              - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${ResourcePrefix}-CID-CURCreator:*"
              - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${ResourcePrefix}-CID-CURCreator:*:*"
        - PolicyName: "ExecutionSpecific"
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
            - Effect: Allow
              Action:
              - cur:PutReportDefinition
              - cur:ModifyReportDefinition
              - cur:DeleteReportDefinition
              Resource:
                - Fn::Sub: arn:${AWS::Partition}:cur:us-east-1:${AWS::AccountId}:definition/*

  CIDLambdaCURCreator:
    Type: AWS::Lambda::Function
    Condition: DeployCURViaLambda
    Properties:
      Runtime: python3.9
      FunctionName:
        Fn::Sub: ${ResourcePrefix}-CID-CURCreator
      Handler: index.lambda_handler
      MemorySize: 128
      Role:
        Fn::GetAtt: CIDLambdaCURCreatorRole.Arn
      Timeout: 15
      Code:
        ZipFile: |
          import boto3
          import cfnresponse
          import uuid
          import json

          # Create a cur client in us-east-1 region
          client = boto3.client('cur', region_name='us-east-1')

          def lambda_handler(event, context):

            print(json.dumps(event))
            reason = ""

            try:
              report = event['ResourceProperties']['ReportDefinition']
              report_name = event['ResourceProperties']['ReportDefinition']['ReportName']

              refresh_closed_report = event['ResourceProperties']['ReportDefinition']["RefreshClosedReports"]
              if refresh_closed_report in ["True", "true"]:
                  report["RefreshClosedReports"] = True
              elif refresh_closed_report in ["False", "false"]:
                  report["RefreshClosedReports"] = False
              else:
                  raise Exception("RefreshClosedReports is not a boolean")

              if event['RequestType'] == 'Create':
                  res = client.put_report_definition(
                      ReportDefinition=report
                  )
                  print(json.dumps(res))
              elif event['RequestType'] == 'Update':
                  old_report_name = event['OldResourceProperties']['ReportDefinition']['ReportName']
                  if report["ReportName"] != old_report_name:
                      res = client.put_report_definition(
                          ReportDefinition=report
                      )
                      print(json.dumps(res))
                  else:
                      res = client.modify_report_definition(
                          ReportName=old_report_name,
                          ReportDefinition=report
                      )
                      print(json.dumps(res))
              elif event['RequestType'] == 'Delete':
                  try:
                      res = client.delete_report_definition(
                          ReportName=report_name
                      )
                      print(json.dumps(res))
                  except:
                      pass # Do not block deletion
              else:
                  raise Exception("Unknown operation: " + event['RequestType'])

            except Exception as e:
                reason = str(e)
                print(e)
            finally:
                physicalResourceId = event.get('ResourceProperties',{}).get('ReportDefinition').get('ReportName', None) or str(uuid.uuid1())
                if reason:
                    print("FAILURE")
                    cfnresponse.send(event, context, cfnresponse.FAILED, {"Data": reason }, physicalResourceId)
                else:
                    print("SUCCESS")
                    cfnresponse.send(event, context, cfnresponse.SUCCESS, {}, physicalResourceId)
tlinkin commented 7 months ago

Please merge !