Unable to update to CUDOS v5

Yeah except in the one we deployed that was linked in the CUDOS lab, there is no typo and the name for the function is the same. I see that the function name has been updated in #678 and then CustomRessourceFunctionInit resource id with the old function name has been added later in #684.
AWSTemplateFormatVersion: '2010-09-09'
Description: Deployment of Cloud Intelligence Dashboards
Metadata:
  AWS::CloudFormation::Interface:
    ParameterGroups:
      - Label:
          default: 'Common Parameters'
        Parameters:
          - PrerequisitesQuickSight
          - PrerequisitesQuickSightPermissions
          - QuickSightUser
      - Label:
          default: CUDOS, Cost-Intelligence-Dashboard and KPI-Dashboard. Require deployment of CUR via CloudFormation (cur-aggregation.yaml) or manually (Dashboard data will appear within 24h after CUR creation).
        Parameters:
          - CURBucketPath
          - DeployCUDOSDashboard
          - DeployCostIntelligenceDashboard
          - DeployKPIDashboard
      - Label:
          default: Trusted Advisor and Compute Optimizer Dashboards. To deploy these two dashboard, you must first deploy the Optimization Data Collection Lab (https://wellarchitectedlabs.com/cost/300_labs/300_optimization_data_collection/)
        Parameters:
          - OptimizationDataCollectionBucketPath
          - DeployTAODashboard
          - DeployComputeOptimizerDashboard
          - PrimaryTagName
          - SecondaryTagName
      - Label:
          default: 'Technical Parameters. Please do not change.'
        Parameters:
          - LakeFormationEnabled
          - AthenaWorkgroup
          - AthenaQueryResultsBucket
          - DatabaseName
          - CURTableName
          - CidVersion
          - GlueDataCatalog
          - Suffix
          - QuickSightDataSourceRoleName
          - QuickSightDataSetRefreshSchedule
          - LambdaLayerBucketPrefix
          - DataBuketsKmsKeyArns
    ParameterLabels:
      PrerequisitesQuickSight:
        default: "I have enabled QuickSight Enterprise Edition AND I have a SPICE capacity in the current region."
      PrerequisitesQuickSightPermissions:
        default: "I understand that I need to manually give Permission to QuickSight to access CUR bucket and Query results bucket. Then manually refresh datasets after deploying this CFN."
      LakeFormationEnabled:
        default: "I have LakeFormation permission model in place for this account & my cfn deployment credentials have administrative rights on LakeFormation"
      QuickSightUser:
        default: "User name of QuickSight user (as displayed in QuickSight admin panel). Dashboards created by this template will be owned by this user."
      CURBucketPath:
        default: "Path to Cost and Usage report"
      DeployCUDOSDashboard:
        default: "Deploy CUDOS Dashboard"
      DeployCostIntelligenceDashboard:
        default: "Deploy CostIntelligenceDashboard"
      DeployKPIDashboard:
        default: "Deploy KPI Dashboard"
      OptimizationDataCollectionBucketPath:
        default: "Path to Optimization Data Collection S3 bucket"
      DeployTAODashboard:
        default: "Deploy TAO Dashboard"
      DeployComputeOptimizerDashboard:
        default: "Deploy Compute Optimizer Dashboard"
      AthenaWorkgroup:
        default: "Athena Workgroup - Please do not change"
      AthenaQueryResultsBucket:
        default: "Athena Query Results Bucket - Please do not change"
      DatabaseName:
        default: "Database Name - Please do not change"
      CURTableName:
        default: "CUR Table Name - Please do not change"
      CidVersion:
        default: "Cid Version - Please do not change"
      Suffix:
        default: "Suffix - Please do not change"
      QuickSightDataSourceRoleName:
        default: "IAM Role Name to be used on QuickSight Datasource Creation (if not provided, the default QuickSight Role will be used)."
      QuickSightDataSetRefreshSchedule:
        default: "QuickSight DataSet Refresh Schedule. Must be a valid cron or empty. If empty refresh will be disabled."
      LambdaLayerBucketPrefix:
        default: "LambdaLayerBucketPrefix - Please do not change"
      GlueDataCatalog:
        default: "Existing Glue Data Catalog"
      DataBuketsKmsKeyArns:
        default: "ARNs of KMS Keys for data bucket. Keep empty if data Buckets are not Encrypted with KMS. Also you can set it to '*'."
      PrimaryTagName:
        Default: "Choose a tag name. Currently used only in Compute Optimizer dashboard."
      SecondaryTagName:
        Default: "Choose a tag name. Currently used only in Compute Optimizer dashboard."
  cfn-lint:
    config:
      ignore_checks:
        - W2001
Parameters:
  PrerequisitesQuickSight:
    Type: String
    Description: See https://quicksight.aws.amazon.com/sn/admin#capacity
    ConstraintDescription: 'Please check in QuickSight that you have at least 10GB of SPICE capacity'
    AllowedPattern: 'yes'
    AllowedValues: ["yes", "no"]
  PrerequisitesQuickSightPermissions:
    Type: String
    Description: See https://quicksight.aws.amazon.com/sn/admin#aws
    ConstraintDescription: 'Please read prerequisites'
    AllowedPattern: 'yes'
    AllowedValues: ["yes", "no"]
  QuickSightUser:
    Type: String
    MinLength: 1
    Default: REPLACE WITH QuickSight USER
    Description: See https://quicksight.aws.amazon.com/sn/admin#users
  QuickSightDataSetRefreshSchedule:
    Type: String
    Default: ''
    Description: 'Cron expression on when to refresh spice datasets via Lambda. Only needed if some difficulities with refresh scheduling via API.'
  QuickSightDataSourceRoleName:
    Type: String
    Default: 'CidQuickSightDataSourceRole'
    Description: "IAM Role Name to be used on QuckSight Datasource Creation. If empty - then the Default QuckSight Role will be used; if provided other existing role, will use that Role; if name equal to 'CidQuickSightDataSourceRole', then a role will be created by this CloudFromation)."
  CURBucketPath:
    Type: String
    MinLength: 3
    Default: 's3://cid-{account_id}-shared/cur/'
    AllowedPattern: '^s3://[a-z0-9](.)+[a-zA-Z0-9/]$'
    Description: "Leave as is if CUR was created with CloudFormation (cur-aggregation.yaml). If it was a manually created CUR, the path entered below must be for the directory that contains the years partition (s3://curbucketname/prefix/curname/curname/). If you're using the defaults, the variable {account_id} will be replaced by current account id automatically, you can leave it as {account_id}."
  AthenaWorkgroup:
    Type: String
    Default: ''
    Description: Leave Empty
  AthenaQueryResultsBucket:
    Type: String
    Default: ''
    Description:  Leave Empty
  DatabaseName:
    Type: String
    Description: Leave Empty
    Default: ''
  CURTableName:
    Type: String
    Default: ''
    Description: Leave Empty
  CidVersion:
    Type: String
    MinLength: 5
    Default: 0.2.29
    Description: A version of CID package
  Suffix:
    Type: String
    Description: Leave Empty. Do not use this Suffix it is not fully supported. For testing purposes only.
    Default: ""
  DeployCUDOSDashboard:
    Type: String
    Description: Deploy CUDOS Dashboard
    Default: "no"
    AllowedValues: ["yes", "no"]
  DeployCostIntelligenceDashboard:
    Type: String
    Description: Deploy Cost Intelligence Dashboard
    Default: "no"
    AllowedValues: ["yes", "no"]
  DeployKPIDashboard:
    Type: String
    Description: Deploy KPI Dashboard
    Default: "no"
    AllowedValues: ["yes", "no"]
  DeployTAODashboard:
    Type: String
    Description: Deploy Trusted Advisor Organizational Dashboard (TAO) - WARNING! Before deploying this dashboard, you need Optimization Data Collection Lab to be installed first https://wellarchitectedlabs.com/cost/300_labs/300_optimization_data_collection/
    Default: "no"
    AllowedValues: ["yes", "no"]
  DeployComputeOptimizerDashboard:
    Type: String
    Description: Deploy Compute Optimizer Dashboard (COD) - WARNING! Before deploying this dashboard, you need Optimization Data Collection Lab to be installed first https://wellarchitectedlabs.com/cost/300_labs/300_optimization_data_collection/
    Default: "no"
    AllowedValues: ["yes", "no"]
  OptimizationDataCollectionBucketPath:
    Type: String
    Description: The S3 path to the bucket created by the Cost Optimization Data Collection Lab. The path will need point to a folder containing /trusted-advisor and/or /compute-optimizer folders. You can leave the variable {account_id} in place, it will be replaced by current account ID automatically.
    Default: "s3://costoptimizationdata{account_id}"
    AllowedPattern: '^s3://[a-zA-Z0-9-_{}/]*$'
  LambdaLayerBucketPrefix:
    Type: String
    Description: An S3 bucket with a Lambda layer
    Default: "aws-managed-cost-intelligence-dashboards"
  GlueDataCatalog:
    Type: String
    Description: Existing Glue Data Catalog
    Default: "AwsDataCatalog"
  DataBuketsKmsKeyArns:
    Type: String
    Description: "ARNs of KMS Keys for data bucket. Keep empty if data Buckets are not Encrypted with KMS. Also you can set it to '*'."
    Default: "*"
  LakeFormationEnabled:
    Type: String
    Description: Choose 'yes' if Lake Formation permission model is in place for the account
    Default: "no"
    AllowedValues: ["yes", "no"]
  PrimaryTagName:
    Type: String
    Description: Choose a tag name for Primary Tag. Can be any Tag name (owner, environment, finops_exception). Currently used only in Compute Optimizer dashboard. Leve as is if not sure.
    Default: "owner"
    MinLength: 1 # cid cmd do not accept empty parameters
    AllowedPattern: "[a-zA-Z0-9_]*"
  SecondaryTagName:
    Type: String
    Description: Choose a tag name for Secondary Tag. Can be any Tag name (owner, environment, finops_exception). Currently used only in Compute Optimizer dashboard. Leve as is if not sure.
    Default: "environment"
    MinLength: 1 # cid cmd do not accept empty parameters
    AllowedPattern: "[a-zA-Z0-9_]*"

Conditions:
  NeedCUDOSDashboard: !Equals [ !Ref DeployCUDOSDashboard, "yes" ]
  NeedCostIntelligenceDashboard: !Equals [ !Ref DeployCostIntelligenceDashboard, "yes" ]
  NeedKPIDashboard: !Equals [ !Ref DeployKPIDashboard, "yes" ]
  NeedTAODashboard: !Equals [ !Ref DeployTAODashboard, "yes" ]
  NeedComputeOptimizerDashboard: !Equals [ !Ref DeployComputeOptimizerDashboard, "yes" ]
  NeedCUR:
    Fn::Or:
      - !Equals [ !Ref DeployCUDOSDashboard, "yes" ]
      - !Equals [ !Ref DeployCostIntelligenceDashboard, "yes" ]
      - !Equals [ !Ref DeployKPIDashboard, "yes" ]
  NeedDataCollectionLab:
    Fn::Or:
      - !Equals [ !Ref DeployTAODashboard, "yes" ]
      - !Equals [ !Ref DeployComputeOptimizerDashboard, "yes" ]
  NeedAthenaWorkgroup:  !Equals [ !Ref AthenaWorkgroup, "" ]
  NeedAthenaQueryResultsBucket:  !Equals [ !Ref AthenaQueryResultsBucket, "" ]
  NeedDatabase:
    Fn::And:
      - !Equals [ !Ref DatabaseName, "" ]
      - Fn::Or:
        - !Condition NeedDataCollectionLab
        - !Condition NeedCUR
  NeedCURTable:
    Fn::And:
      - !Equals [ !Ref CURTableName, "" ]
      - !Condition NeedCUR
  NeedRefreshDatasets:  !Not [ !Equals  [ !Ref QuickSightDataSetRefreshSchedule, ""] ]
  NeedDataBucketsKms: !Equals [ !Ref DataBuketsKmsKeyArns, "" ]
  NeedDataBucketsKmsAndNeedCURTable:
    Fn::And:
      - !Condition NeedDataBucketsKms
      - !Condition NeedCURTable
  NeedDatasource: !Not [ !Equals [ !Ref "AWS::Region", "eu-west-3" ] ] # In eu-west-3 CFN QS Dataset resource is not availble yet.
  NeedLakeFormationEnabled:
    Fn::And:
      - !Equals [ !Ref LakeFormationEnabled, "yes" ]
      - Fn::Or:
        - !Equals [ !Ref DeployCUDOSDashboard, "yes" ]
        - !Equals [ !Ref DeployCostIntelligenceDashboard, "yes" ]
        - !Equals [ !Ref DeployKPIDashboard, "yes" ]
  NeedLakeFormationCrawlerPermissions:
    Fn::And:
      - !Equals [ !Ref LakeFormationEnabled, "yes" ]
      - !Condition NeedCURTable
  UseQuickSightDataSourceRole:
    Fn::And:
      - !Condition NeedDatasource
      - !Not [!Equals [ !Ref QuickSightDataSourceRoleName, "" ]]
  NeedQuickSightDataSourceRole:
    Fn::And:
      - !Condition NeedDatasource
      - !Equals [ !Ref QuickSightDataSourceRoleName, "CidQuickSightDataSourceRole" ]
  NeedQuickSightDataSourceRoleAndCUR:
    Fn::And:
      - !Condition NeedQuickSightDataSourceRole
      - !Condition NeedCUR
  NeedQuickSightDataSourceRoleAndODC:
    Fn::And:
      - !Condition NeedQuickSightDataSourceRole
      - !Condition NeedDataCollectionLab

Resources:
  SpiceRefreshExecutionRole: #Role needed to schedule spice ingestion for the datasets
    Type: AWS::IAM::Role
    Condition: NeedRefreshDatasets
    Properties:
      Path: /
      RoleName: !Sub 'CidSpiceRefreshExecutionRole${Suffix}'
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      ManagedPolicyArns:
        - !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
        - PolicyName: !Sub 'CidSpiceRefreshExecutionRole${Suffix}'
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action: quicksight:CreateIngestion
                Resource:
                  - !Sub 'arn:${AWS::Partition}:quicksight:${AWS::Region}:${AWS::AccountId}:*'
              - Effect: Allow
                Action: quicksight:ListDatasets
                Resource:
                  - !Sub 'arn:${AWS::Partition}:quicksight:${AWS::Region}:${AWS::AccountId}:dataset/*'
              - Effect: Allow
                Action: quicksight:ListIngestions
                Resource:
                  - !Sub 'arn:${AWS::Partition}:quicksight:${AWS::Region}:${AWS::AccountId}:dataset/*/ingestion/*'

  # Currently QS has no api for managing updates, so we need to set up a scheduled lambda.
  # Once QS will provide the API for scheduling this will be removed.
  SpiceRefreshLambda:
    Type: AWS::Lambda::Function
    Condition: NeedRefreshDatasets
    Properties:
      FunctionName:  !Sub 'CidSpiceRefreshLambda${Suffix}'
      Role: !GetAtt SpiceRefreshExecutionRole.Arn
      Description: 'Refresh QuickSight DataSets for CID'
      Runtime: python3.9
      Architectures: [ x86_64 ] #Compatible with arm64 but it is not supported in all regions
      MemorySize: 128
      Timeout: 60
      Environment:
        Variables:
          #SUFFIX: !Ref Suffix
          SUFFIX: '' # CID CMD does not support suffixes yet
      Handler: index.lambda_handler
      Code:
        ZipFile: |
          import os
          from datetime import datetime
          from datetime import timedelta
          from datetime import timezone
          import boto3

          ## List of DataSets can be found in cid-cmd:
          # from cid.common import Cid
          # DATASETS = list(Cid().resources['datasets'].keys())

          DATASETS = '''
            summary_view
            ec2_running_cost
            compute_savings_plan_eligible_spend
            s3_view
            kpi_ebs_snap
            kpi_ebs_storage_all
            kpi_instance_all
            kpi_s3_storage_all
            kpi_tracker
            ta-organizational-view
            daily-anomaly-detection
            monthly-anomaly-detection
            monthly-bill-by-account
            compute_optimizer_all_options
          '''.strip().split()

          DATASETS_TO_REFRESH = [ name + os.environ.get('SUFFIX', '') for name in DATASETS]

          def lambda_handler(event, context):
              account_id = context.invoked_function_arn.split(":")[4]
              quicksight = boto3.client('quicksight')
              for page in quicksight.get_paginator('list_data_sets').paginate(AwsAccountId=account_id):
                  for dataset in page['DataSetSummaries']:
                      name = dataset['Name']
                      if dataset['ImportMode'] != 'SPICE' or name not in DATASETS_TO_REFRESH:
                          continue
                      scheduled_ingestion = None
                      stop_processing = False
                      for ingestions_page in quicksight.get_paginator('list_ingestions').paginate(AwsAccountId=account_id, DataSetId=dataset['DataSetId']):
                          for ingestion in ingestions_page['Ingestions']:
                              time_since_creation = datetime.now(timezone.utc) - ingestion['CreatedTime']
                              if time_since_creation <= timedelta(days = 1, hours = 1):
                                  if ingestion['RequestSource'] == 'SCHEDULED':
                                      scheduled_ingestion = ingestion
                                      stop_processing = True
                              else:
                                  stop_processing = True
                              if stop_processing:
                                  break
                          if stop_processing:
                              break
                      if scheduled_ingestion is not None:
                          print(f"INFO: Dataset {name} has a scheduled ingestion within the last 24 hours. Skipping manual refresh.")
                          print('DEBUG: scheduled_ingestion=', scheduled_ingestion)
                          continue
                      print(f"INFO: Refreshing dataset {name}")
                      res = quicksight.create_ingestion(
                          AwsAccountId=account_id,
                          DataSetId=dataset['DataSetId'],
                          IngestionId=datetime.now().strftime("%d%m%y-%H%M%S-%f"),
                      )
                      print('DEBUG: response=', res)

  SpiceRefreshRule:
    Type: AWS::Events::Rule
    Condition: NeedRefreshDatasets
    Properties:
      ScheduleExpression: !Ref QuickSightDataSetRefreshSchedule
      Targets:
        - Id: SpiceRefreshScheduler
          Arn: !GetAtt SpiceRefreshLambda.Arn

  SpiceRefreshInvokeLambdaPermission:
    Type: AWS::Lambda::Permission
    Condition: NeedRefreshDatasets
    Properties:
      FunctionName: !GetAtt SpiceRefreshLambda.Arn
      Action: lambda:InvokeFunction
      Principal: events.amazonaws.com
      SourceArn: !GetAtt SpiceRefreshRule.Arn

  MyAthenaQueryResultsBucket:
    Type: AWS::S3::Bucket
    Condition: NeedAthenaQueryResultsBucket
    Properties:
      BucketName: !Sub "${AWS::Partition}-athena-query-results-cid-${AWS::AccountId}-${AWS::Region}"
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256
      AccessControl: BucketOwnerFullControl
      OwnershipControls:
        Rules:
          - ObjectOwnership: BucketOwnerEnforced
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true
      LifecycleConfiguration:
        Rules:
          - Id: DeleteContent
            Status: 'Enabled'
            ExpirationInDays: 7
    Metadata:
      cfn-lint:
        config:
          ignore_checks:
            - W3045 #Consider using AWS::S3::BucketPolicy instead of AccessControl; standard Athena results setup

  MyAthenaWorkGroup:
    Type: AWS::Athena::WorkGroup
    Condition: NeedAthenaWorkgroup
    Properties:
      Name: !Sub 'CID${Suffix}'
      Description: !Sub 'Used for CloudIntelligenceDashboards${Suffix}'
      WorkGroupConfiguration:
        EnforceWorkGroupConfiguration: true
        ResultConfiguration:
          EncryptionConfiguration:
            EncryptionOption: SSE_S3
          OutputLocation: !If [ NeedAthenaQueryResultsBucket, !Sub 's3://${MyAthenaQueryResultsBucket}/', !Sub 's3://${AthenaQueryResultsBucket}/' ]

  CustomResourceFunctionInit:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: !Sub "CidInitialSetup-DoNotRun${Suffix}"
      Role: !GetAtt 'InitLambdaExecutionRole.Arn'
      Description: "Do what CFN cannot: start crawler, delete bucket with objects and delete an non empty workgroup"
      Runtime: python3.9
      Architectures: [ x86_64 ] #Compatible with arm64 but it is not supported in all regions
      MemorySize: 128
      Timeout: 300
      Handler: 'index.lambda_handler'
      Code:
        ZipFile: |
          import os
          import uuid
          import json
          import boto3
          import botocore
          import urllib3

          from cid.helpers import QuickSight # from layer
          from cid.utils import set_parameters

          BUCKET = os.environ['BUCKET']
          WORKGROUP = os.environ['WORKGROUP']
          CRAWLER = os.environ['CRAWLER']
          QUICKSIGHT_USER = os.environ['QUICKSIGHT_USER']

          def lambda_handler(event, context):
              print(event)
              type_ = event.get('RequestType', 'Undef')
              region = boto3.session.Session().region_name
              res = (True, f"Un error on {type_}. Check logs")
              identity_region = ''
              try:
                  if type_ == 'Create': res = on_create()
                  elif type_ == 'Delete': res = on_delete()
                  else: res = (True, f"Not supported operation: {type_}")
                  set_parameters({'quicksight-user': QUICKSIGHT_USER})
                  identity_region = get_identity_region()
              finally:
                  log_url = f"https://{region}.console.aws.amazon.com/cloudwatch/home?region={region}#logEvent:group={context.log_group_name};stream={context.log_stream_name}"
                  url = event.get('ResponseURL')
                  body = {}
                  body['Status'] = 'SUCCESS' if res[0] else 'FAILED'
                  body['Reason'] = res[1] + '\nLogs: ' + log_url
                  body['PhysicalResourceId'] = 'keep_it_constant'
                  body['StackId'] = event.get('StackId')
                  body['RequestId'] = event.get('RequestId')
                  body['LogicalResourceId'] = event.get('LogicalResourceId')
                  body['NoEcho'] = False
                  body['Data'] =  {'Reason': res[1], 'uuid': str(uuid.uuid1()), 'IdentityRegion': identity_region}
                  print(body)
                  if not url: return
                  json_body=json.dumps(body)
                  try:
                      http = urllib3.PoolManager()
                      response = http.request('PUT', url, body=json_body, headers={'content-type' : '', 'content-length' : str(len(json_body))}, retries=False)
                      print(f"Status code: {response}")
                  except Exception as exc:
                      print("Failed sending PUT to CFN: " + str(exc))

          def get_identity_region():
              qs = QuickSight(boto3.session.Session())
              return qs.identityRegion

          def on_create():
              if CRAWLER:
                  try:
                      boto3.client('glue').start_crawler(Name=CRAWLER)
                  except Exception as exc:
                      return (True, f'ERROR: error invoking crawler {CRAWLER} {exc}')
                  return (True, 'INFO: crawler started. Takes 1 min to update the table.')
              return (True, 'INFO: No actions on create')

          def on_delete():
              # Delete bucket (CF cannot delete if they are non-empty)
              # and delete WorkGroup (CF cannot do that)
              s3 = boto3.resource('s3')
              log = []

              if BUCKET:
                  try:
                      bucket = s3.Bucket(BUCKET)
                      res = bucket.object_versions.delete()
                      print(f'DEBUG: empty response = {res} ')
                      res = bucket.delete()
                      print(f'DEBUG: delete response = {res} ')
                      log.append(f'INFO:  {BUCKET} deleted')
                  except botocore.exceptions.ClientError as exc:
                      status = exc.response["ResponseMetadata"]["HTTPStatusCode"]
                      errcode = exc.response["Error"]["Code"]
                      if status == 404:
                          log.append(f'INFO:  {BUCKET} - {errcode}')
                      else:
                          log.append(f'ERROR: {BUCKET} - {errcode}')
                  except Exception as exc:
                      log.append(f'ERROR: {BUCKET} Error: {exc}')

              if WORKGROUP:
                  try:
                      response = boto3.client('athena').delete_work_group(
                          WorkGroup=WORKGROUP,
                          RecursiveDeleteOption=True
                      )
                      print(f'DEBUG: WorkGroup {WORKGROUP} deleted. {response}')
                      log.append(f'INFO: WorkGroup {WORKGROUP} deleted.')
                  except botocore.exceptions.ClientError as exc:
                      status = exc.response["ResponseMetadata"]["HTTPStatusCode"]
                      errcode = exc.response["Error"]["Code"]
                      if status == 404:
                          log.append(f'INFO:  WorkGroup {WORKGROUP} - {errcode}')
                      else:
                          log.append(f'ERROR: WorkGroup {WORKGROUP} - {errcode}')
                  except Exception as exc:
                      log.append(f'ERROR: WorkGroup {WORKGROUP} Error: {exc}')
              print('\n'.join(log))
              return (True, '\n'.join(log))
      Layers:
        - !Ref CidResourceLambdaLayer
      Environment:
        Variables:
          BUCKET: !If [NeedAthenaQueryResultsBucket, !Ref MyAthenaQueryResultsBucket, '']
          WORKGROUP: !If [NeedAthenaWorkgroup, !Ref MyAthenaWorkGroup, '']
          CRAWLER: !If [NeedCURTable, !Ref MyGlueCURCrawler, '']
          QUICKSIGHT_USER: !Ref QuickSightUser

  InitLambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action: quicksight:DescribeUser
                Resource: !Sub 'arn:${AWS::Partition}:quicksight:*:${AWS::AccountId}:user/default/${QuickSightUser}' # region=* as at this moment we do not know the Identity region where QS stores users
      ManagedPolicyArns:
        - !Sub arn:${AWS::Partition}:iam::aws:policy/AWSLambdaExecute

  InitLambdaExecutionRoleWorkGroupPoliciy:
    Type: AWS::IAM::Policy
    Condition: NeedAthenaWorkgroup
    Properties:
      PolicyName: AthenaWorkGroupDeletion
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action: athena:DeleteWorkGroup
            Resource: !Sub 'arn:${AWS::Partition}:athena:${AWS::Region}:${AWS::AccountId}:workgroup/${MyAthenaWorkGroup}'
      Roles:
        - !Ref InitLambdaExecutionRole

  InitLambdaExecutionRoleBucketPoliciy:
    Type: AWS::IAM::Policy
    Condition: NeedAthenaQueryResultsBucket
    Properties:
      PolicyName: AthenaQueryResultsBucketDeletion
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action:
              - s3:DeleteObject
              - s3:DeleteObjectVersion
            Resource:
              - !Sub 'arn:${AWS::Partition}:s3:::${MyAthenaQueryResultsBucket}/*'
          - Effect: Allow
            Action:
              - s3:ListBucketVersions
              - s3:DeleteBucket
            Resource:
              - !Sub 'arn:${AWS::Partition}:s3:::${MyAthenaQueryResultsBucket}'
      Roles:
        - !Ref InitLambdaExecutionRole

  InitLambdaExecutionRoleStartCrawlerPoliciy:
    Type: AWS::IAM::Policy
    Condition: NeedCURTable
    Properties:
      PolicyName: StartCrawler
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action:
              - glue:StartCrawler
            Resource:
              - !Sub 'arn:${AWS::Partition}:glue:${AWS::Region}:${AWS::AccountId}:crawler/${MyGlueCURCrawler}'
      Roles:
        - !Ref InitLambdaExecutionRole

  InitialSetup:
    Type: Custom::CustomResource
    Properties:
      ServiceToken: !GetAtt CustomResourceFunctionInit.Arn
      Tags: # Hacky way to manage conditional dependencies
        - Key: IgnoreConditionalDependsOnAthenaQueryResultsBucket
          Value: !If [NeedAthenaQueryResultsBucket, !Ref MyAthenaQueryResultsBucket, '']
        - Key: IgnoreConditionalDependsOnAthenaWorkgroup
          Value: !If [NeedAthenaWorkgroup, !Ref MyAthenaWorkGroup, '']
        - Key: IgnoreConditionalDependsOnDatabase
          Value: !If [NeedCURTable, !Ref MyGlueCURCrawler, '']
        - Key: IgnoreConditionalDependsOnPolicy1
          Value: !If [NeedAthenaWorkgroup, !Ref InitLambdaExecutionRoleWorkGroupPoliciy, '']
        - Key: IgnoreConditionalDependsOnPolicy2
          Value: !If [NeedAthenaQueryResultsBucket, !Ref InitLambdaExecutionRoleBucketPoliciy, '']
        - Key: IgnoreConditionalDependsOnPolicy3
          Value: !If [NeedCURTable, !Ref InitLambdaExecutionRoleStartCrawlerPoliciy, '']

  ProcessPathLambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      ManagedPolicyArns:
        - !Sub arn:${AWS::Partition}:iam::aws:policy/AWSLambdaExecute
  CustomResourceProcessPath:
    Type: AWS::Lambda::Function
    Properties:
      Role: !GetAtt 'ProcessPathLambdaExecutionRole.Arn'
      FunctionName: !Sub "CidProcessPath-DoNotRun${Suffix}"
      Description: "Do what CFN cannot: process string of path"
      Runtime: python3.9
      Architectures: [ x86_64 ] #Compatible with arm64 but it is not supported in all regions
      MemorySize: 128
      Timeout: 60
      Handler: 'index.lambda_handler'
      Code:
        ZipFile: |
          import uuid
          import json
          import urllib3
          import botocore
          import boto3

          partitions = {
            "managed_by_cfn": ["source_account_id", "cur_name_1", "cur_name_2", "year", "month"],
            "manual":         ["year", "month"],
          }

          def lambda_handler(event, context):
              print(json.dumps(event))
              account_id = context.invoked_function_arn.split(":")[4]
              type_ = event.get('RequestType', 'Undef')
              region = boto3.session.Session().region_name
              properties = event.get('ResourceProperties', {})
              status, reason = ('SUCCESS', "Undef")
              data = {}
              body = {}
              try:
                  s3path = properties.get('s3path', '')
                  if s3path.startswith('s3://'):
                      s3path = s3path[len('s3://'):]
                  if s3path.endswith('/'):
                      s3path = s3path[:-1]
                  s3path = s3path.replace('{account_id}', account_id)
                  parts = s3path.split('/')
                  data['Bucket'] = parts[0]
                  if not bucket_exists(data['Bucket']) and type_.lower() != 'delete':
                      raise Exception(f'Bucket {parts[0]} does not exist. Please check prerequisites. Just creating a bucket is not enough.')
                  if properties.get('type', '') == 'CUR':
                      # detect Type of CUR and choose the right partitions structure
                      if len(parts[1:]) == 1: # most likely it is created by CFN or similar
                          data['Partitions'] = partitions['managed_by_cfn']
                      elif len(parts) > 3 and parts[-1] == parts[-2]: # most likely it is manual CUR
                          data['Partitions'] = partitions['manual']
                      elif type_.lower() == 'delete':
                          pass # Do not fail delete
                      else:
                          raise Exception(f'CUR BucketPath={parts[0]} format is not recognized. It must be s3://(bucket)/cur or s3://(bucket)/(curprefix)/(curname)/(curname) ')
                      data['Partitions'] = [{"Name": p, "Type": "string"} for p in data['Partitions']]
                  data['Path'] = '/'.join(parts[1:])
                  data['Folder'] = parts[-1] if len(parts) > 1 else ''
                  data['Folder'] = data['Folder'].replace('-', '_').lower() # this is used for a Glue table name that will be managed by crawler
                  status, reason = 'SUCCESS', ""
              except Exception as exc:
                  status, reason = 'FAILED', str(exc)
              finally:
                  log_url = f"https://{region}.console.aws.amazon.com/cloudwatch/home?region={region}#logEvent:group={context.log_group_name};stream={context.log_stream_name}"
                  url = event.get('ResponseURL')
                  body['Status'] = status
                  body['Reason'] = reason + '\nLogs: ' + log_url
                  body['PhysicalResourceId'] = s3path
                  body['StackId'] = event.get('StackId')
                  body['RequestId'] = event.get('RequestId')
                  body['LogicalResourceId'] = event.get('LogicalResourceId')
                  body['Data'] = data
                  json_body=json.dumps(body)
                  print(json_body)
                  if not url: return
                  try:
                      http = urllib3.PoolManager()
                      response = http.request('PUT', url, body=json_body, headers={'content-type' : '', 'content-length' : str(len(json_body))}, retries=False)
                      print(f"Status code: {response}")
                  except Exception as exc:
                      print("Failed sending PUT to CFN: " + str(exc))

          def bucket_exists(name):
              try:
                  boto3.resource('s3').meta.client.head_bucket(Bucket=name)
              except botocore.exceptions.ClientError as e:
                  if e.response['Error']['Code'] == '404':
                      return False
              return True

  ProcessedCURPath:
    Type: Custom::CustomResourceProcessPath
    Condition: NeedCUR
    Properties:
      ServiceToken: !GetAtt CustomResourceProcessPath.Arn
      s3path: !Ref CURBucketPath
      type: 'CUR'

  ProcessedODCPath:
    Type: Custom::CustomResourceProcessPath
    Condition: NeedDataCollectionLab
    Properties:
      ServiceToken: !GetAtt CustomResourceProcessPath.Arn
      s3path: !Ref OptimizationDataCollectionBucketPath

  CidDatabase:
    Type: AWS::Glue::Database
    Condition: NeedDatabase
    Properties:
      DatabaseInput:
        Name: !Join [ '_', !Split [ '-', !Sub 'cid_cur${Suffix}' ] ] # replace '-' to '_'
      CatalogId: !Sub '${AWS::AccountId}'

  MyGlueCURCrawler:
    Type: AWS::Glue::Crawler
    Condition: NeedCURTable
    Properties:
      Name:  !Sub 'CidCrawler${Suffix}'
      Description: A recurring crawler that keeps your CUR table in Athena up-to-date.
      Role:
        Fn::GetAtt: CidCURCrawlerRole.Arn
      DatabaseName: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]
      Targets:
        S3Targets:
          - Path: !Sub 's3://${ProcessedCURPath.Bucket}/${ProcessedCURPath.Path}/'
            Exclusions:
              - '**.json'
              - '**.yml'
              - '**.sql'
              - '**.csv'
              - '**.csv.metadata'
              - '**.gz'
              - '**.zip'
              - '**/cost_and_usage_data_status/*'
              - 'aws-programmatic-access-test-object'
      SchemaChangePolicy:
        DeleteBehavior: LOG
      RecrawlPolicy:
        RecrawlBehavior: CRAWL_EVERYTHING
      Schedule:
        ScheduleExpression: cron(0 2 * * ? *)
      Configuration: |
        {
          "Version":1.0,
          "Grouping": {
            "TableGroupingPolicy": "CombineCompatibleSchemas"
          },
          "CrawlerOutput":{
            "Tables":{
              "AddOrUpdateBehavior":"MergeNewColumns"
            }
          }
        }

  MyCURTable:  # Initial creation of table. it will be updated by crawler later
    Type: AWS::Glue::Table
    Condition: NeedCURTable
    Properties:
      CatalogId: !Ref "AWS::AccountId"
      DatabaseName: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]
      TableInput:
        Name: !GetAtt ProcessedCURPath.Folder
        Owner: owner
        Retention: 0
        TableType: EXTERNAL_TABLE
        Parameters:
          compressionType: none
          classification: parquet
          UPDATED_BY_CRAWLER: !Ref MyGlueCURCrawler
        StorageDescriptor:
          BucketColumns: []
          Compressed: false
          Location: !Sub 's3://${ProcessedCURPath.Bucket}/${ProcessedCURPath.Path}/'
          NumberOfBuckets: -1
          InputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat
          OutputFormat: org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
          SerdeInfo:
            Parameters:
              serialization.format: '1'
            SerializationLibrary: org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe
          StoredAsSubDirectories: false
          Columns: # All fields required for CID
            - {"Name": "bill_bill_type", "Type": "string" }
            - {"Name": "bill_billing_entity", "Type": "string" }
            - {"Name": "bill_billing_period_end_date", "Type": "timestamp" }
            - {"Name": "bill_billing_period_start_date", "Type": "timestamp" }
            - {"Name": "bill_invoice_id", "Type": "string" }
            - {"Name": "bill_payer_account_id", "Type": "string" }
            - {"Name": "identity_line_item_id", "Type": "string" }
            - {"Name": "identity_time_interval", "Type": "string" }
            - {"Name": "line_item_availability_zone", "Type": "string" }
            - {"Name": "line_item_legal_entity", "Type": "string" }
            - {"Name": "line_item_line_item_description", "Type": "string" }
            - {"Name": "line_item_line_item_type", "Type": "string" }
            - {"Name": "line_item_operation", "Type": "string" }
            - {"Name": "line_item_product_code", "Type": "string" }
            - {"Name": "line_item_resource_id", "Type": "string" }
            - {"Name": "line_item_unblended_cost", "Type": "double" }
            - {"Name": "line_item_usage_account_id", "Type": "string" }
            - {"Name": "line_item_usage_amount", "Type": "double" }
            - {"Name": "line_item_usage_end_date", "Type": "timestamp" }
            - {"Name": "line_item_usage_start_date", "Type": "timestamp" }
            - {"Name": "line_item_usage_type", "Type": "string" }
            - {"Name": "pricing_lease_contract_length", "Type": "string" }
            - {"Name": "pricing_offering_class", "Type": "string" }
            - {"Name": "pricing_public_on_demand_cost", "Type": "double" }
            - {"Name": "pricing_purchase_option", "Type": "string" }
            - {"Name": "pricing_unit", "Type": "string" }
            - {"Name": "product_cache_engine", "Type": "string" }
            - {"Name": "product_current_generation", "Type": "string" }
            - {"Name": "product_database_engine", "Type": "string" }
            - {"Name": "product_deployment_option", "Type": "string" }
            - {"Name": "product_from_location", "Type": "string" }
            - {"Name": "product_group", "Type": "string" }
            - {"Name": "product_instance_type", "Type": "string" }
            - {"Name": "product_instance_type_family", "Type": "string" }
            - {"Name": "product_license_model", "Type": "string" }
            - {"Name": "product_operating_system", "Type": "string" }
            - {"Name": "product_physical_processor", "Type": "string" }
            - {"Name": "product_processor_features", "Type": "string" }
            - {"Name": "product_product_family", "Type": "string" }
            - {"Name": "product_product_name", "Type": "string" }
            - {"Name": "product_region", "Type": "string" }
            - {"Name": "product_servicecode", "Type": "string" }
            - {"Name": "product_tenancy", "Type": "string" }
            - {"Name": "product_to_location", "Type": "string" }
            - {"Name": "product_volume_api_name", "Type": "string" }
            - {"Name": "product_volume_type", "Type": "string" }
            - {"Name": "reservation_amortized_upfront_fee_for_billing_period", "Type": "double" }
            - {"Name": "reservation_effective_cost", "Type": "double" }
            - {"Name": "reservation_end_time", "Type": "string" }
            - {"Name": "reservation_reservation_a_r_n", "Type": "string" }
            - {"Name": "reservation_start_time", "Type": "string" }
            - {"Name": "reservation_unused_amortized_upfront_fee_for_billing_period", "Type": "double" }
            - {"Name": "reservation_unused_recurring_fee", "Type": "double" }
            - {"Name": "savings_plan_amortized_upfront_commitment_for_billing_period", "Type": "double" }
            - {"Name": "savings_plan_end_time", "Type": "string" }
            - {"Name": "savings_plan_offering_type", "Type": "string" }
            - {"Name": "savings_plan_payment_option", "Type": "string" }
            - {"Name": "savings_plan_purchase_term", "Type": "string" }
            - {"Name": "savings_plan_savings_plan_a_r_n", "Type": "string" }
            - {"Name": "savings_plan_savings_plan_effective_cost", "Type": "double" }
            - {"Name": "savings_plan_start_time", "Type": "string" }
            - {"Name": "savings_plan_total_commitment_to_date", "Type": "double" }
            - {"Name": "savings_plan_used_commitment", "Type": "double" }
        PartitionKeys: !GetAtt ProcessedCURPath.Partitions

  CidCURCrawlerRole:
    Type: AWS::IAM::Role
    Condition: NeedCURTable
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - glue.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      ManagedPolicyArns:
        - Fn::Sub: 'arn:${AWS::Partition}:iam::aws:policy/service-role/AWSGlueServiceRole'
      Policies:
        - PolicyName: AWSCURCrawlerComponentFunction
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'glue:UpdateDatabase'
                  - 'glue:UpdatePartition'
                  - 'glue:CreateTable'
                  - 'glue:UpdateTable'
                  - 'glue:ImportCatalogToGlue'
                Resource:
                  - !Sub arn:${AWS::Partition}:glue:${AWS::Region}:${AWS::AccountId}:catalog
                  - Fn::If:
                      - NeedDatabase
                      - !Sub arn:${AWS::Partition}:glue:${AWS::Region}:${AWS::AccountId}:database/${CidDatabase}
                      - !Sub arn:${AWS::Partition}:glue:${AWS::Region}:${AWS::AccountId}:database/${DatabaseName}
                  - Fn::If:
                      - NeedDatabase
                      - !Sub arn:${AWS::Partition}:glue:${AWS::Region}:${AWS::AccountId}:table/${CidDatabase}/${ProcessedCURPath.Folder}
                      - !Sub arn:${AWS::Partition}:glue:${AWS::Region}:${AWS::AccountId}:table/${DatabaseName}/${ProcessedCURPath.Folder}
              - Effect: Allow
                Action:
                  - 's3:GetObject'
                Resource:
                  Fn::Sub: 'arn:${AWS::Partition}:s3:::${ProcessedCURPath.Bucket}/${ProcessedCURPath.Path}/*'

  KmsPoliciyForCidCURCrawlerRole:
    Type: AWS::IAM::Policy
    Condition: NeedDataBucketsKmsAndNeedCURTable
    Properties:
      PolicyName: AwsCurCrawlerKmsDecryption
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action:
              - 'kms:Decrypt'
            Resource: !Split [',', !Ref DataBuketsKmsKeyArns]
      Roles:
        - !Ref CidCURCrawlerRole

  QuickSightDataSourceRole:
    Type: AWS::IAM::Role
    Condition: NeedQuickSightDataSourceRole
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - quicksight.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /
      RoleName: !Sub '${QuickSightDataSourceRoleName}${Suffix}'
      Policies:
        - PolicyName: AthenaAccess
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - lakeformation:GetDataAccess
                  - athena:ListDataCatalogs
                  - athena:ListDatabases
                  - athena:ListTableMetadata
                Resource: "*" # required https://docs.aws.amazon.com/lake-formation/latest/dg/access-control-underlying-data.html
                              # Cannot restrict this. See https://docs.aws.amazon.com/athena/latest/ug/datacatalogs-example-policies.html#datacatalog-policy-listing-data-catalogs
              - Effect: Allow
                Action:
                  - glue:GetPartitions
                  - glue:GetDatabases
                  - glue:GetTable
                  - glue:GetTables
                Resource:
                  - !Sub 'arn:${AWS::Partition}:glue:${AWS::Region}:${AWS::AccountId}:catalog'
                  - Fn::If:
                     - NeedDatabase
                     - !Sub arn:${AWS::Partition}:glue:${AWS::Region}:${AWS::AccountId}:database/${CidDatabase}
                     - !Sub arn:${AWS::Partition}:glue:${AWS::Region}:${AWS::AccountId}:database/${DatabaseName}
                  - Fn::If:
                     - NeedDatabase
                     - !Sub arn:${AWS::Partition}:glue:${AWS::Region}:${AWS::AccountId}:table/${CidDatabase}/*
                     - !Sub arn:${AWS::Partition}:glue:${AWS::Region}:${AWS::AccountId}:table/${DatabaseName}/*
              - Effect: Allow
                Action:
                  - athena:ListDatabases
                  - athena:ListDataCatalogs
                  - athena:ListDatabases
                  - athena:GetQueryExecution
                  - athena:GetQueryResults
                  - athena:StartQueryExecution
                  - athena:GetQueryResultsStream
                  - athena:ListTableMetadata
                  - athena:GetTableMetadata
                Resource:
                  - Fn::If:
                      - NeedDatabase
                      - !Sub arn:${AWS::Partition}:athena:${AWS::Region}:${AWS::AccountId}:database/${CidDatabase}
                      - !Sub arn:${AWS::Partition}:athena:${AWS::Region}:${AWS::AccountId}:database/${DatabaseName}
                  - !Sub 'arn:${AWS::Partition}:athena:${AWS::Region}:${AWS::AccountId}:datacatalog/${GlueDataCatalog}'
                  - Fn::If:
                     - NeedAthenaWorkgroup
                     - !Sub 'arn:${AWS::Partition}:athena:${AWS::Region}:${AWS::AccountId}:workgroup/${MyAthenaWorkGroup}'
                     - !Sub 'arn:${AWS::Partition}:athena:${AWS::Region}:${AWS::AccountId}:workgroup/${AthenaWorkgroup}'
              - Effect: Allow
                Action:
                  - s3:GetBucketLocation
                  - s3:ListBucket
                  - s3:GetObject
                  - s3:PutObject
                  - s3:ListBucketMultipartUploads
                  - s3:ListMultipartUploadParts
                  - s3:AbortMultipartUpload
                Resource:
                  - Fn::If:
                      - NeedAthenaQueryResultsBucket
                      - !Sub 'arn:${AWS::Partition}:s3:::${MyAthenaQueryResultsBucket}'
                      - !Sub 'arn:${AWS::Partition}:s3:::${AthenaQueryResultsBucket}'
                  - Fn::If:
                      - NeedAthenaQueryResultsBucket
                      - !Sub 'arn:${AWS::Partition}:s3:::${MyAthenaQueryResultsBucket}/*'
                      - !Sub 'arn:${AWS::Partition}:s3:::${AthenaQueryResultsBucket}/*'

  QuickSightDataSourceRolePolicyForODCBucket:
    Type: AWS::IAM::Policy
    Condition: NeedQuickSightDataSourceRoleAndODC
    Properties:
      PolicyName: QuickSightDataSource-S3AccessODC
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Sid: CidAllowDecryptDataBuketsKmsKeyArns
            Effect: Allow
            Action: 'kms:Decrypt'
            Resource: !Split [',', !Ref DataBuketsKmsKeyArns]
          - Sid: CidAllowListBucket
            Effect: Allow
            Action: s3:ListBucket
            Resource: !Sub arn:aws:s3:::${ProcessedODCPath.Bucket}
          - Sid: CidAllowReadBucket
            Effect: Allow
            Action:
              - s3:GetObject
              - s3:GetObjectVersion
            Resource: !Sub arn:aws:s3:::${ProcessedODCPath.Bucket}/*
      Roles:
        - !Ref QuickSightDataSourceRole
  QuickSightDataSourceRolePolicyForCURBucket:
    Type: AWS::IAM::Policy
    Condition: NeedQuickSightDataSourceRoleAndCUR
    Properties:
      PolicyName: QuickSightDataSource-S3AccessCUR
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Sid: CidAllowDecryptDataBuketsKmsKeyArns
            Effect: Allow
            Action: 'kms:Decrypt'
            Resource: !Split [',', !Ref DataBuketsKmsKeyArns]
          - Sid: CidAllowListBucket
            Effect: Allow
            Action: s3:ListBucket
            Resource: !Sub arn:aws:s3:::${ProcessedCURPath.Bucket}
          - Sid: CidAllowReadBucket
            Effect: Allow
            Action:
              - s3:GetObject
              - s3:GetObjectVersion
            Resource: !Sub arn:aws:s3:::${ProcessedCURPath.Bucket}/*
      Roles:
        - !Ref QuickSightDataSourceRole

  CidAthenaDataSource:
    Type: AWS::QuickSight::DataSource
    Condition: NeedDatasource
    Properties:
      AwsAccountId: !Sub '${AWS::AccountId}'
      Type: ATHENA
      DataSourceId: !Sub 'CID-Athena${Suffix}'
      Name:         !Sub 'CID-Athena${Suffix}'
      DataSourceParameters:
        AthenaParameters:
          WorkGroup: !If [ NeedAthenaWorkgroup, !Ref MyAthenaWorkGroup, !Ref AthenaWorkgroup ]
          RoleArn: !If [ UseQuickSightDataSourceRole, !Sub "arn:aws:iam::${AWS::AccountId}:role/${QuickSightDataSourceRoleName}", !Ref AWS::NoValue ]
      Permissions:
        - Actions:
            - 'quicksight:DescribeDataSource'
            - 'quicksight:DescribeDataSourcePermissions'
            - 'quicksight:PassDataSource'
            - 'quicksight:UpdateDataSource'
            - 'quicksight:DeleteDataSource'
            - 'quicksight:UpdateDataSourcePermissions'
          Principal: !Sub 'arn:${AWS::Partition}:quicksight:${InitialSetup.IdentityRegion}:${AWS::AccountId}:user/default/${QuickSightUser}'

  CidExecRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      RoleName: !Sub 'CidExecRole${Suffix}'
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      ManagedPolicyArns:
        - !Sub arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
        - PolicyName: CidExecPolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - athena:GetWorkGroup
                Resource:
                  Fn::If:
                   - NeedAthenaWorkgroup
                   - !Sub 'arn:${AWS::Partition}:athena:${AWS::Region}:${AWS::AccountId}:workgroup/${MyAthenaWorkGroup}'
                   - !Sub 'arn:${AWS::Partition}:athena:${AWS::Region}:${AWS::AccountId}:workgroup/${AthenaWorkgroup}'
              - Effect: Allow
                Action:
                  - glue:GetDatabase
                  - glue:GetDatabases
                  - glue:GetTable
                  - glue:UpdateTable
                  - glue:GetTables
                  - glue:GetPartitions
                  - glue:CreateTable
                Resource: "*" # This is needed to allow Autodetection in CID-CMD
              - Effect: Allow
                Action:
                  - s3:ListBucket
                  - s3:ListBucketMultipartUploads
                  - s3:ListMultipartUploadParts
                  - s3:AbortMultipartUpload
                  - s3:GetBucketLocation
                  - s3:GetObject
                  - s3:PutObject
                Resource:
                  - Fn::If:
                    - NeedAthenaQueryResultsBucket
                    - !Sub 'arn:${AWS::Partition}:s3:::${MyAthenaQueryResultsBucket}'
                    - !Sub 'arn:${AWS::Partition}:s3:::${AthenaQueryResultsBucket}'
                  - Fn::If:
                    - NeedAthenaQueryResultsBucket
                    - !Sub 'arn:${AWS::Partition}:s3:::${MyAthenaQueryResultsBucket}/*'
                    - !Sub 'arn:${AWS::Partition}:s3:::${AthenaQueryResultsBucket}/*'
              - Effect: Allow
                Action:
                  - quicksight:DescribeDataSource
                  - quicksight:CreateDataSource
                  - quicksight:DeleteDataSource
                Resource: # only needed if CFN for Datasource is not available
                    - !Sub 'arn:${AWS::Partition}:quicksight:${AWS::Region}:${AWS::AccountId}:datasource/CID-Athena-1'
              - Effect: Allow
                Action:
                  - quicksight:ListDataSources #
                  - quicksight:ListDataSets
                  - quicksight:CreateDataSet
                  - quicksight:DescribeDataSet
                  - quicksight:DeleteDataSet
                  - quicksight:UpdateDataSet
                  - quicksight:PassDataSource #
                  - quicksight:PassDataSet
                  - quicksight:UpdateDataSetPermissions
                  - quicksight:CreateDashboard
                  - quicksight:DescribeDashboard
                  - quicksight:DeleteDashboard
                  - quicksight:UpdateDashboard
                  - quicksight:UpdateDashboardPermissions
                  - quicksight:UpdateDashboardPublishedVersion
                  - quicksight:ListDashboards
                  - quicksight:DescribeUser
                  - quicksight:DescribeTemplate
                  - quicksight:DescribeAccountSubscription
                Resource: '*' # This is needed to allow Autodetection in CID-CMD
              - Effect: Allow
                Action:
                - quicksight:CreateRefreshSchedule
                - quicksight:UpdateRefreshSchedule
                - quicksight:DeleteRefreshSchedule
                - quicksight:DescribeRefreshSchedule
                - quicksight:ListRefreshSchedules
                Resource:
                - !Sub arn:aws:quicksight:${AWS::Region}:${AWS::AccountId}:dataset/* # DataSetIDs are dynamic as well as schedule ids
              - Effect: Allow
                Action:
                  - athena:StartQueryExecution
                  - athena:GetQueryResults
                  - athena:GetQueryExecution
                  - athena:GetTableMetadata
                  - athena:ListDatabases
                  - athena:ListDataCatalogs
                  - athena:ListEngineVersions
                  - athena:ListTableMetadata
                  - athena:ListWorkGroups
                  - athena:GetDatabase
                Resource: '*'  # This is needed to allow Autodetection in CID-CMD

  DataLakeSettingsCidExecRolePerm:
    Type: AWS::LakeFormation::Permissions
    Condition: NeedLakeFormationEnabled
    Properties:
      DataLakePrincipal:
        DataLakePrincipalIdentifier: !GetAtt CidExecRole.Arn
      Permissions:
        - ALL
      Resource:
        DatabaseResource:
          CatalogId: !Ref "AWS::AccountId"
          Name: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]

  DataLakeSettingsCidExecRolePermTable:
    Type: AWS::LakeFormation::Permissions
    Condition: NeedLakeFormationEnabled
    Properties:
      DataLakePrincipal:
        DataLakePrincipalIdentifier: !GetAtt CidExecRole.Arn
      Permissions:
        - ALL
      Resource:
        TableResource:
          CatalogId: !Ref "AWS::AccountId"
          DatabaseName: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]
          TableWildcard: {}

  DataLakeSettingsCidCrawlerRolePerm:
    Type: AWS::LakeFormation::Permissions
    Condition: NeedLakeFormationCrawlerPermissions
    Properties:
      DataLakePrincipal:
        DataLakePrincipalIdentifier: !GetAtt CidCURCrawlerRole.Arn
      Permissions:
        - ALL
      Resource:
        DatabaseResource:
          CatalogId: !Ref "AWS::AccountId"
          Name: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]

  DataLakeSettingsCidCrawlerRolePermTable:
    Type: AWS::LakeFormation::Permissions
    Condition: NeedLakeFormationCrawlerPermissions
    Properties:
      DataLakePrincipal:
        DataLakePrincipalIdentifier: !GetAtt CidCURCrawlerRole.Arn
      Permissions:
        - ALL
      Resource:
        TableResource:
          CatalogId: !Ref "AWS::AccountId"
          DatabaseName: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]
          TableWildcard: {}

  DataLakeSettingQuickSightAdminUserPerm:
    Type: AWS::LakeFormation::Permissions
    Condition: NeedLakeFormationEnabled
    Properties:
      DataLakePrincipal:
        DataLakePrincipalIdentifier: !Sub 'arn:${AWS::Partition}:quicksight:${InitialSetup.IdentityRegion}:${AWS::AccountId}:user/default/${QuickSightUser}'
      Permissions:
        - ALL
      Resource:
        DatabaseResource:
          CatalogId: !Ref "AWS::AccountId"
          Name: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]

  DataLakeSettingQuickSightAdminUserPermTable:
    Type: AWS::LakeFormation::Permissions
    Condition: NeedLakeFormationEnabled
    Properties:
      DataLakePrincipal:
        DataLakePrincipalIdentifier: !Sub 'arn:${AWS::Partition}:quicksight:${InitialSetup.IdentityRegion}:${AWS::AccountId}:user/default/${QuickSightUser}'
      Permissions:
        - ALL
      Resource:
        TableResource:
          CatalogId: !Ref "AWS::AccountId"
          DatabaseName: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]
          TableWildcard: {}

  KmsPoliciyForCidExecRole:
    Type: AWS::IAM::Policy
    Condition: NeedDataBucketsKms
    Properties:
      PolicyName: AwsCurCrawlerKmsDecryption
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action:
              - 'kms:Decrypt'
            Resource: !Split [',', !Ref DataBuketsKmsKeyArns]
      Roles:
        - !Ref CidExecRole

  CidExec: #custom lambda resource that deploy views, datasets and dashboards
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: !Sub 'CidCustomResourceDashboard${Suffix}'
      Description: 'A lambda that manage create delete update of Athena views, QuickSight Datasets and dashboards using CID-CMD tool'
      Role: !GetAtt CidExecRole.Arn
      Runtime: python3.9
      Architectures: [ x86_64 ] #Compatible with arm64 but it is not supported in all regions
      MemorySize: 2688
      Timeout: 300 # Time of discovery depend on number of dashboards
      Handler: index.lambda_handler
      Code:
        ZipFile: |
          import os
          import uuid
          import json
          import logging

          import boto3
          import requests
          from cid.common import Cid # From lambda layer
          from cid.utils import set_parameters # From lambda layer
          from cid.exceptions import CidCritical

          Cid._Cid__setupLogging = lambda self, verbosity: None #Monkey patch to avoid file creation

          logger = logging.getLogger(__name__)
          logger.setLevel(logging.INFO)
          logging.getLogger('cid').setLevel(logging.DEBUG)

          account_id = boto3.client('sts').get_caller_identity()['Account']
          region = boto3.session.Session().region_name

          def lambda_handler(event, context):
              print(json.dumps(event))
              request_type = event.get('RequestType', 'Undef')
              properties = event.get('ResourceProperties', {})
              status, reason = ('FAILED', "Undef error")
              physical_id = properties.get('Dashboard', {}).get('dashboard-id', 'Unknown')
              dash_url = "Unknown"
              try:
                  dashboard = properties['Dashboard']
                  if request_type == 'Create':
                      dash_url = deploy_dash(dashboard)
                      status, reason = 'SUCCESS', f"{request_type} {physical_id} ok"
                  elif request_type == 'Delete':
                      dash_url = delete_dash(dashboard)
                      status, reason = 'SUCCESS', f"{request_type} {physical_id} ok"
                  elif request_type == 'Update':
                      dash_url = update_dash(dashboard)
                      status, reason = 'SUCCESS', f"{request_type} {physical_id} ok"
                  else:
                      status, reason = 'SUCCESS', f"Not supported operation: {request_type}"
              except Exception as exc:
                  logger.exception(exc)
                  status, reason = ('FAILED', f"Failed {request_type} {physical_id} with exception: {exc}.")
              except CidCritical as exc:
                  logger.debug(exc, exc_info=True)
                  status, reason = ('FAILED', f"{exc}")
              except SystemExit as exc:
                  status, reason = ('FAILED', f"Cid called exit({exc.code}) on {request_type} {physical_id}")
              finally:
                  log_url = f"https://{region}.console.aws.amazon.com/cloudwatch/home?region={region}#logEvent:group={context.log_group_name};stream={context.log_stream_name}"
                  url = event.get('ResponseURL')
                  body = {
                      'Status': status,
                      'Reason': reason[:2000] + '\nSee more: ' + log_url,
                      'PhysicalResourceId': physical_id,
                      'StackId': event.get('StackId'),
                      'RequestId': event.get('RequestId'),
                      'LogicalResourceId': event.get('LogicalResourceId'),
                      'NoEcho': False,
                      'Data': {'Reason': reason, 'DashboardURL': dash_url },
                  }
                  json_body = json.dumps(body)
                  print(json_body)
                  if not url: return
                  try:
                      res = requests.put(url, data=json_body, headers={'content-type' : '','content-length' : str(len(json_body))})
                      print(f"return {res.status_code}: {res.text}" )
                  except Exception as exc:
                      print("send(..) failed executing requests.put(..): " + str(exc))

          def deploy_dash(params):
              app = Cid(verbose=3)
              set_parameters(params, all_yes=True)
              app.deploy()
              return app.qs_url.format(dashboard_id=params['dashboard-id'], **app.qs_url_params)

          def delete_dash(params):
              app = Cid(verbose=3)
              set_parameters(params, all_yes=True)
              app.delete(dashboard_id=params['dashboard-id'])
              return ''

          def update_dash(params):
              app = Cid(verbose=3)
              set_parameters(params, all_yes=True)
              app.update(dashboard_id=params['dashboard-id'])
              return app.qs_url.format(dashboard_id=params['dashboard-id'], **app.qs_url_params)
      Layers:
        - !Ref CidResourceLambdaLayer

  CidResourceLambdaLayer:
    Type: AWS::Lambda::LayerVersion
    Properties:
      LayerName: !Sub 'CidLambdaLayer${Suffix}'
      Description: An AWS managed layer with a cid-cmd package installed
      Content:
        S3Bucket: !Sub '${LambdaLayerBucketPrefix}-${AWS::Region}'
        S3Key: !Sub 'cid-resource-lambda-layer/cid-${CidVersion}.zip'
      CompatibleRuntimes:
        - python3.9

  CostIntelligenceDashboard:
    Type: Custom::CidDashboard
    Condition: NeedCostIntelligenceDashboard
    DependsOn:
      - InitialSetup
    Properties:
      Name: !Sub 'CloudIntelligenceDashboard${Suffix}'
      ServiceToken: !GetAtt CidExec.Arn
      Dashboard:
        dashboard-id: cost_intelligence_dashboard
        athena-workgroup: !If [ NeedAthenaWorkgroup, !Ref MyAthenaWorkGroup, !Ref AthenaWorkgroup ]
        quicksight-datasource-id: !If [ NeedDatasource, !Select [ 1, !Split [ '/', !GetAtt CidAthenaDataSource.Arn]], 'CID-Athena-1']
        quicksight-datasource-role-arn: !If [ NeedQuickSightDataSourceRole, !Sub "arn:aws:iam::${AWS::AccountId}:role/${QuickSightDataSourceRole}", "" ]
        athena-database: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]
        glue-data-catalog: !Ref GlueDataCatalog
        cur-table-name: !If [ NeedCURTable, !Ref MyCURTable, !Ref CURTableName ]
        quicksight-user: !Ref QuickSightUser
        account-map-source: 'dummy' #initial
        share-with-account: 'yes'

  CUDOSDashboard:
    Type: Custom::CidDashboard
    Condition: NeedCUDOSDashboard
    DependsOn:
      - InitialSetup
    Properties:
      Name: !Sub 'CUDOSDashboard${Suffix}'
      ServiceToken: !GetAtt CidExec.Arn
      Dashboard:
        dashboard-id: cudos
        athena-workgroup: !If [ NeedAthenaWorkgroup, !Ref MyAthenaWorkGroup, !Ref AthenaWorkgroup ]
        quicksight-datasource-id: !If [ NeedDatasource, !Select [ 1, !Split [ '/', !GetAtt CidAthenaDataSource.Arn]], 'CID-Athena-1']
        quicksight-datasource-role-arn: !If [ NeedQuickSightDataSourceRole, !Sub "arn:aws:iam::${AWS::AccountId}:role/${QuickSightDataSourceRole}", "" ]
        athena-database: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]
        glue-data-catalog: !Ref GlueDataCatalog
        cur-table-name: !If [ NeedCURTable, !Ref MyCURTable, !Ref CURTableName ]
        quicksight-user: !Ref QuickSightUser
        account-map-source: 'dummy' #initial
        share-with-account: 'yes'
      Tags: # Hacky way to manage conditional dependencies
        - Key: IgnoreNeedCostIntelligenceDashboard
          Value: !If [NeedCostIntelligenceDashboard, !Ref CostIntelligenceDashboard, '']

  KPIDashboard:
    Type: Custom::CidDashboard
    Condition: NeedKPIDashboard
    DependsOn:
      - InitialSetup
    Properties:
      Name: !Sub 'KPIDashboard${Suffix}'
      ServiceToken: !GetAtt CidExec.Arn
      Dashboard:
        dashboard-id: kpi_dashboard
        athena-workgroup: !If [ NeedAthenaWorkgroup, !Ref MyAthenaWorkGroup, !Ref AthenaWorkgroup ]
        quicksight-datasource-id: !If [ NeedDatasource, !Select [ 1, !Split [ '/', !GetAtt CidAthenaDataSource.Arn]], 'CID-Athena-1']
        quicksight-datasource-role-arn: !If [ NeedQuickSightDataSourceRole, !Sub "arn:aws:iam::${AWS::AccountId}:role/${QuickSightDataSourceRole}", "" ]
        athena-database: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]
        glue-data-catalog: !Ref GlueDataCatalog
        cur-table-name: !If [ NeedCURTable, !Ref MyCURTable, !Ref CURTableName ]
        quicksight-user: !Ref QuickSightUser
        account-map-source: 'dummy' #initial
        share-with-account: 'yes'
      Tags: # Hacky way to manage conditional dependencies
        - Key: IgnoreNeedCostIntelligenceDashboard
          Value: !If [NeedCostIntelligenceDashboard, !Ref CostIntelligenceDashboard, '']
        - Key: IgnoreNeedCUDOSDashboard
          Value: !If [NeedCUDOSDashboard, !Ref CUDOSDashboard, '']

  TAODashboard:
    Type: Custom::CidDashboard
    Condition: NeedTAODashboard
    DependsOn:
      - InitialSetup
    Properties:
      Name: !Sub 'TAODashboard${Suffix}'
      ServiceToken: !GetAtt CidExec.Arn
      Dashboard:
        dashboard-id: ta-organizational-view
        athena-workgroup: !If [ NeedAthenaWorkgroup, !Ref MyAthenaWorkGroup, !Ref AthenaWorkgroup ]
        quicksight-datasource-id: !If [ NeedDatasource, !Select [ 1, !Split [ '/', !GetAtt CidAthenaDataSource.Arn]], 'CID-Athena-1']
        quicksight-datasource-role-arn: !If [ NeedQuickSightDataSourceRole, !Sub "arn:aws:iam::${AWS::AccountId}:role/${QuickSightDataSourceRole}", "" ]
        athena-database: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]
        glue-data-catalog: !Ref GlueDataCatalog
        cur-table-name: !If [ NeedCURTable, !Ref MyCURTable, !Ref CURTableName ]
        quicksight-user: !Ref QuickSightUser
        share-with-account: 'yes'
        view-ta-organizational-view-reports-s3FolderPath: !Sub '${OptimizationDataCollectionBucketPath}/trusted-advisor/trusted-advisor-data'

  ComputeOptimizerDashboard:
    Type: Custom::CidDashboard
    Condition: NeedComputeOptimizerDashboard
    DependsOn:
      - InitialSetup
    Properties:
      Name: !Sub 'ComputeOptimizerDashboard${Suffix}'
      ServiceToken: !GetAtt CidExec.Arn
      Dashboard:
        dashboard-id: compute-optimizer-dashboard
        athena-workgroup: !If [ NeedAthenaWorkgroup, !Ref MyAthenaWorkGroup, !Ref AthenaWorkgroup ]
        quicksight-datasource-id: !If [ NeedDatasource, !Select [ 1, !Split [ '/', !GetAtt CidAthenaDataSource.Arn]], 'CID-Athena-1']
        quicksight-datasource-role-arn: !If [ NeedQuickSightDataSourceRole, !Sub "arn:aws:iam::${AWS::AccountId}:role/${QuickSightDataSourceRole}", "" ]
        athena-database: !If [NeedDatabase, !Ref CidDatabase, !Ref DatabaseName ]
        glue-data-catalog: !Ref GlueDataCatalog
        cur-table-name: !If [ NeedCURTable, !Ref MyCURTable, !Ref CURTableName ]
        quicksight-user: !Ref QuickSightUser
        share-with-account: 'yes'
        view-compute-optimizer-lambda-lines-s3FolderPath:       !Sub '${OptimizationDataCollectionBucketPath}/compute_optimizer/compute_optimizer_lambda'
        view-compute-optimizer-ebs-volume-lines-s3FolderPath:   !Sub '${OptimizationDataCollectionBucketPath}/compute_optimizer/compute_optimizer_ebs_volume'
        view-compute-optimizer-auto-scale-lines-s3FolderPath:   !Sub '${OptimizationDataCollectionBucketPath}/compute_optimizer/compute_optimizer_auto_scale'
        view-compute-optimizer-ec2-instance-lines-s3FolderPath: !Sub '${OptimizationDataCollectionBucketPath}/compute_optimizer/compute_optimizer_ec2_instance'
        dataset-compute-optimizer-all-options-primary-tag-name: !Sub '${PrimaryTagName}'
        dataset-compute-optimizer-all-options-secondary-tag-name: !Sub '${SecondaryTagName}'

Outputs:
  CostIntelligenceDashboardURL:
    Description: "URL of CostIntelligenceDashboard"
    Condition: NeedCostIntelligenceDashboard
    Value: !GetAtt CostIntelligenceDashboard.DashboardURL
  CUDOSDashboardURL:
    Description: "URL of CUDOSDashboard"
    Condition: NeedCUDOSDashboard
    Value: !GetAtt CUDOSDashboard.DashboardURL
  KPIDashboardURL:
    Description: "URL of KPIDashboard"
    Condition: NeedKPIDashboard
    Value: !GetAtt KPIDashboard.DashboardURL
  TAODashboardURL:
    Description: "URL of TAODashboard"
    Condition: NeedTAODashboard
    Value: !GetAtt TAODashboard.DashboardURL
  ComputeOptimizerDashboardURL:
    Description: "URL of ComputeOptimizerDashboard"
    Condition: NeedComputeOptimizerDashboard
    Value: !GetAtt ComputeOptimizerDashboard.DashboardURL
aws-samples / aws-cudos-framework-deployment

Unable to update to CUDOS v5 #756
